[{"content":"My polished notes for whenever I\u0026rsquo;m learning about something new and I think it\u0026rsquo;s worth sharing. Designed as a personal incentive structure (I doomscroll).\n","date":"29 August 2026","externalUrl":null,"permalink":"/learning/","section":"Learning","summary":"","title":"Learning","type":"learning"},{"content":"","date":"29 August 2026","externalUrl":null,"permalink":"/series/llms/","section":"Series","summary":"","title":"LLMs","type":"series"},{"content":"","date":"29 August 2026","externalUrl":null,"permalink":"/","section":"Mostly Second Thoughts","summary":"","title":"Mostly Second Thoughts","type":"page"},{"content":"","date":"29 August 2026","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"This is a test\n","date":"29 August 2026","externalUrl":null,"permalink":"/learning/test/","section":"Learning","summary":"shudezihd","title":"Test","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/series/learning-llms/","section":"Series","summary":"","title":"Learning LLMs","type":"series"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/categories/llms/","section":"Categories","summary":"","title":"LLMs","type":"categories"},{"content":"What I\u0026rsquo;ve had to work through to stop hand-waving. Filed here before I forget why I bothered.\n","date":"28 August 2026","externalUrl":null,"permalink":"/learning/llms/","section":"Learning","summary":"Notes on large language models: the working through, and the parts I keep having to re-derive.","title":"LLMs","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/tags/meta/","section":"Tags","summary":"","title":"Meta","type":"tags"},{"content":" Before writing anything worth reading, it seems prudent to check that the site can display it. If inline math typeset by KaTeX, a centred display equation, a syntax-highlighted Python snippet, and a captioned figure all render cleanly below, the plumbing is in order and future posts can concern themselves with their contents rather than their appearance.\nInline math inside prose # Scaled dot-product attention computes a soft lookup from queries \\(Q\\), keys \\(K\\), and values \\(V\\). Written inline, it is \\(\\text{Attention}(Q, K, V) = \\text{softmax}\\!\\left(\\tfrac{QK^{\\top}}{\\sqrt{d_k}}\\right) V\\), where the \\(\\sqrt{d_k}\\) divisor keeps the pre-softmax logits from growing too large as the key dimension \\(d_k\\) increases.\nA display equation # Language models are trained to minimize the token-level cross-entropy loss. For a sequence of tokens \\(x_1, \\dots, x_T\\), this is\n$$ \\mathcal{L}(\\theta) = -\\frac{1}{T} \\sum_{t=1}^{T} \\log p_\\theta\\!\\left(x_t \\mid x_{\\lt t}\\right). $$Each term \\(\\log p_\\theta(x_t \\mid x_{\\lt t})\\) is the log-probability the model assigns to the true next token given everything that came before.\nA code block # Here is a compact PyTorch-flavoured implementation of the attention scores from the equation above. Nothing fancy, just enough to see syntax highlighting work.\nimport math import torch import torch.nn.functional as F def attention(Q, K, V, mask=None): \u0026#34;\u0026#34;\u0026#34;Scaled dot-product attention.\u0026#34;\u0026#34;\u0026#34; d_k = Q.size(-1) scores = Q @ K.transpose(-2, -1) / math.sqrt(d_k) if mask is not None: scores = scores.masked_fill(mask == 0, float(\u0026#34;-inf\u0026#34;)) weights = F.softmax(scores, dim=-1) return weights @ V, weights A figure with caption # Below is a 4-token causal attention pattern after softmax. The lower-triangular structure reflects the mask: token \\(t\\) can only attend to tokens \\(\\le t\\).\nA 4x4 causal attention matrix: each row sums to 1, upper triangle is zero because of the causal mask. That\u0026rsquo;s it. If everything above reads cleanly in both light and dark mode, the site is calibrated correctly.\n","date":"28 August 2026","externalUrl":null,"permalink":"/learning/llms/hello-math-test/","section":"Learning","summary":"A load-bearing sanity check for the site: does the maths render, and does the code sit next to it without complaint.","title":"Proof that the math renders","type":"learning"},{"content":"","date":"28 August 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"My name is Clément and this website is where I\u0026rsquo;m going to write down everything I would otherwise bury and forget about, in public for accountability. I started Mostly Second Thoughts in order to make sure my drafts wouldn\u0026rsquo;t die out and my notes would get an outlet. It\u0026rsquo;s meant to span from what I\u0026rsquo;m learning quite formally to my opinion on random and/or niche topics. The idea is that the prospect of publishing an actual piece for everyone to see will force me to engage actively with the concepts I\u0026rsquo;m trying to grasp by explaining them. Another equally important goal is to learn to better express myself and get these damn ideas out of my head. Fun fact, writing has been shown to reduce cognitive load for this exact reason.\nIf you feel inspired or have anything to say, feel free to contact me for any reason whatsoever, I\u0026rsquo;ve left some links below that you can click for that purpose. You can also suggest a new topic or angle for me to think about for a next piece in a dedicated section of this website.\nCurrently: I\u0026rsquo;ve been using LLMs extensively for both professional and personal use and I\u0026rsquo;m quite fascinated by the technology and its level of advancement. I\u0026rsquo;ve taken on a path to learn about these algorithms progressively, with the objective of understanding them in depth and being able to read and understand complex architectures and advanced research papers. I am currently building an understanding starting with basic machine learning and neural network concepts, and going back to the associated maths whenever I\u0026rsquo;m lacking any kind of prerequisite.\n","externalUrl":null,"permalink":"/about/","section":"About","summary":"","title":"About","type":"about"},{"content":" Suggest something I should learn about, or a topic that deserves to be pondered. I keep a list for when I\u0026rsquo;m bored. Website Your suggestion Who are you? (optional, Anon) Send Received. Your suggestion joins a queue I take more seriously than it looks. That didn\u0026rsquo;t send. Try again in a bit; the server is usually more cooperative the second time. ","externalUrl":null,"permalink":"/suggest/","section":"Mostly Second Thoughts","summary":"","title":"Suggest a topic","type":"page"},{"content":"Random stuff I have an angle on. Think around and find out I guess.\n","externalUrl":null,"permalink":"/thoughtpieces/","section":"Thoughtpieces","summary":"","title":"Thoughtpieces","type":"thoughtpieces"}]