Frontier Research Patterns

Past the technical content, becoming a frontier AI researcher requires meta-skills that are rarely taught explicitly. How do you read a paper? Not the same way you read a textbook — papers are written for an audience that already has context, and reading them naively is slow and frustrating. The standard three-pass method (Keshav, 2007) — bibliographic scan, structural reading, deep critical read — is the canonical reading methodology. When should you skim a paper versus deep-read it? Skim almost everything; deep-read only the papers whose ideas you intend to build on or argue against. How do you replicate a result? The first replication you attempt will fail, and the second, and the third — replicating papers is its own skill, and the GitHub-to-paper handshake (read the paper, read the code, run the code, compare to the reported numbers, find the discrepancy, debug both) is the central craft. The state of replication in ML is poor; many published results do not reproduce, and identifying which results are real is a core researcher skill. How do you stay current? Twitter (X) is still the primary venue for AI research dissemination despite the field's complaints; alphaxiv and arxiv-sanity provide curated feeds; Hacker News and Reddit's r/MachineLearning serve as discussion venues. How do you choose problems? Read everything Schmidhuber once said about Schmidhuber, then ignore the style; but on the substance, the question of which problems are tractable, important, and neglected is the core researcher question. How do you write a paper? Read Jennifer Widom's writing advice. Read papers you admire and study their structure. By the end of this path you should be able to consume a hundred arXiv papers a week and extract signal from them, replicate a published result end-to-end, identify open problems worth working on, and write a paper that other researchers would actually want to read.

::reading path · in order

1. ::01 · paper

   ~1h

   How to Read a Paper — S. Keshav (ACM SIGCOMM Computer Communication Review, 2007)

   The three-pass paper-reading method. Two pages. Read it before reading anything else.

2. ::02 · lecture

   ~1h

   Andrew Ng — How to Read Research Papers (career advice talk, Stanford)

   Practical paper-reading and field-entry advice. Watch for the breadth-first reading strategy.

3. ::03 · blog

   ~2h

   Andrej Karpathy — A Recipe for Training Neural Networks (blog post)

   How an expert actually debugs a deep learning project. Required reading on craft.

4. ::04 · blog

   ~4h

   Sebastian Raschka — Ahead of AI newsletter (substack)

   Curated weekly summary of significant AI research. Useful for staying current and developing taste for what matters.

5. ::05 · blog

   ~4h

   Jack Clark — Import AI newsletter (importai.substack.com)

   Long-running weekly newsletter from an Anthropic cofounder. Both technical and policy coverage.

6. ::06 · blog

   ~10h

   Lilian Weng — Lil'Log (lilianweng.github.io)

   Long-form deep-dive blog posts on major topics. Read selectively when you need a topic summary.

7. ::07 · blog

   ~1h

   Jennifer Widom — Tips for Writing Technical Papers

   Stanford CS professor's classic writing-advice document. Short and useful.

8. ::08 · code

   ~5h

   Papers with Code (paperswithcode.com)

   Paper-to-code links, benchmark leaderboards. Useful for finding replications and reference implementations.

9. ::09 · paper

   ~4h

   Yann LeCun, Geoffrey Hinton, Yoshua Bengio — Deep Learning (Nature 2015 review)

   Historical perspective from the field's founders. Read for the broad map and the field's own narrative of itself.

10. ::10 · paper

    ~12h

    Patrick Kidger — On Neural Differential Equations (PhD thesis, 2022)

    An example of a well-written modern ML PhD thesis. Read for the writing structure and the methodology of literature integration.

11. ::11 · blog

    ~2h

    arxiv-sanity-lite (arxiv-sanity-lite.com) and alphaxiv (alphaxiv.org)

    Tools for tracking and discussing recent arXiv preprints. Use them weekly.

12. ::12 · blog

    ~15h

    Distill.pub archive (distill.pub)

    Now-archived journal of interactive ML explanations. The clarity bar to aspire to in your own writing.