From 6 items, 2 important content pieces were selected
Technology News
- Simon Willison's Annotated Keynote Recaps 2026 LLM Developments ⭐️ 7.0/10
- NeurIPS Paper Claims Adaptive Functional Gradient Descent Beats Neural Nets ⭐️ 7.0/10
Technology News
Simon Willison's Annotated Keynote Recaps 2026 LLM Developments ⭐️ 7.0/10
Simon Willison published an annotated version of the closing keynote he gave at the WeAreDevelopers World Congress North America in San Jose on 25 September 2026, tying the year's LLM trends into a chronological tour; the post, dated 27 September 2026, links to the talk video on YouTube. In the portion of the deck supplied, Willison dates the start of his 2026 to November 2025, when Claude Opus 4.5 and GPT-5.1 arrived as incremental model improvements that nonetheless crossed an invisible line: paired with their coding agent harnesses — Claude Code, around since February 2025, and the slightly younger Codex — they moved from "often make mistakes" to "reliable enough to use on a day-to-day basis." He illustrates the November state of the art with his long-running "Generate an SVG of a pelican riding a bicycle" test, noting that Claude still could not really draw a bicycle and the GPT-5.1 frame was poor, and he flags the 24 November 2025 first commit to an obscure GitHub repository called "Warelay" as something he would return to later. He also describes his 2026 New Year's resolution to "be more ambitious" and take on as many new projects as he likes, and lists predictions that it will become undeniable that LLMs write good code, that sandboxing will finally be solved, and that there will be a "Challenger disaster" for coding agent security. The supplied excerpt cuts off partway through the predictions slide, so the talk's later technical detail and specific claims cannot be verified from the provided content.
rss · Simon Willison · Sep 27, 23:54
「Background」 The post is an "annotated presentation": the slides from Simon Willison's closing keynote at the WeAreDevelopers World Congress North America in San Jose on 25 September 2026, published alongside his speaker notes as a chronological tour of LLM developments. The talk treats 2026 as having begun in November 2025, when Claude Opus 4.5 and GPT-5.1 shipped as incremental model upgrades that, paired with their respective coding-agent harnesses (Claude Code, available since February 2025, and the newer Codex), crossed from "often make mistakes" to "reliable enough to use on a day-to-day basis." Willison's "generate an SVG of a pelican riding a bicycle" prompt is his long-running, self-described "world's stupidest benchmark" for informally eyeballing what new models can and cannot draw.
「Impact」 For developers already using Claude Code and Codex, the November 2025 pairing of Claude Opus 4.5 and GPT-5.1 with those harnesses shifted agentic coding from "often make mistakes" to "reliable enough to use on a day-to-day basis," which is why Willison reversed his usual "stay focused" resolution and deliberately took on more projects in 2026. He cautions that unlocking the full potential of these agents still requires extraordinary discipline and knowledge, so the widened scope of work comes with a matching rise in engineering demands.
References
Tags: #LLMs, #AI trends, #annotated talk, #Simon Willison, #2026 retrospective
NeurIPS Paper Claims Adaptive Functional Gradient Descent Beats Neural Nets ⭐️ 7.0/10
A first-author Reddit post announced that the paper “Functional Gradient Descent with Adaptive Representations” has been accepted at NeurIPS. The post says functional gradient descent algorithms generally outperform neural nets but are hard to implement accurately because functional gradients are infinite-dimensional and must be approximated, and naive approximations converge to the wrong place. To address this, the authors formalize a broad class of approximation schemes called “adaptive representations,” which they claim provably ensure convergence to the global minimizer while being immediately implementable. The resulting algorithms are said to outperform corresponding neural nets often by an order of magnitude across several settings, though the author describes the work as a starting point with potential. The linked arXiv paper is 2606.16926, and commenters noted caveats, including that the global-optimality claim requires a functional convexity condition (Polyak-Lojasiewicz) and that practicality may be limited to low-dimensional problems.
reddit · r/MachineLearning · dccsillag0 · Sep 28, 13:23 · Discussion
「Background」 Functional gradient descent (FGD) performs gradient descent directly in function space instead of optimizing the parameters of a fixed representation such as a neural network, which yields highly nonconvex losses that complicate training and theoretical analysis, while FGD itself benefits from strong convergence results and a clean theory. Because functional gradients are infinite-dimensional, they must be approximated in practice, and naive approximations can converge to the wrong solution. The paper "Functional Gradient Descent with Adaptive Representations" (arXiv:2606.16926, dated June 15, 2026) formalizes a broad class of approximation schemes, termed adaptive representations, that are claimed to provably ensure convergence to the global minimizer while remaining immediately implementable.
「Impact」 Researchers and practitioners evaluating functional gradient methods may gain an implementable framework with claimed order-of-magnitude gains over comparable neural nets, but its global-convergence guarantee appears conditional on a functional convexity assumption and its practical dimensionality limits remain unresolved.
「Community Discussion」 Commenters were largely positive but raised caveats: one said the global-optimality claim should mention the required Polyak-Lojasiewicz functional convexity condition, another questioned whether the approach is practical only for low-dimensional problems, and a third asked whether it is a new optimizer comparable to Adamw or Muon. The supplied thread does not include author answers to those questions.
References
Tags: #machine learning, #optimization, #functional gradient descent, #neural networks, #NeurIPS