In this tutorial, we build and run a Colab workflow for Gemma 3 1B Instruct using Hugging Face Transformers and HF Token, in a practical, reproducible, and easy-to-follow step-by-step manner. We begin by installing the required libraries, securely authenticating with our Hugging Face token, and loading the tokenizer and model onto the available device with the correct precision settings. From there, we create reusable generation utilities, format prompts in a chat-style structure, and test the model across multiple […]

Ver mais

arXiv:2505.15443v2 Announce Type: replace-cross Abstract: Uncertainty estimation remains a key challenge when adapting pre-trained language models to downstream classification tasks, with overconfidence often observed for difficult inputs. While predictive entropy provides a strong baseline for uncertainty estimation, it considers mainly aleatoric uncertainty and has limited capacity to capture effects, such as class overlap or ambiguous linguistic cues. We introduce Aligned Entropy – ALIEN, a lightweight method that refines entropy-based uncertainty by aligning it with prediction reliability. ALIEN trains […]

Ver mais

We did experiments comparing Optuna & autoresearch. Autoresearch converges faster, is more cost-efficient, and even generalizes better. Experiments were done on NanoChat: we let Claude define Optuna’s search space to align the priors between methods. Both optimization methods were run three times. Autoresearch is far more sample-efficient on average In 5 min training setting, LLM tokens cost as much as GPUs, but despite a 2× higher per-step cost, AutoResearch still comes out ahead across all cost budgets: What’s […]

Ver mais

Class-incremental learning (CIL) with pre-trained models (PTMs) faces a critical trade-off between efficient adaptation and long-term stability. While analytic learning enables rapid, recursive closed-form updates, its efficacy is often compromised by accumulated errors and feature incompatibility. In this paper, we first conduct a systematic study to dissect the failure modes of PTM-based analytic CIL, identifying representation rigidity as the primary bottleneck. Motivated by these insights, we propose textbf{VILA}, a novel dual-branch framework that advances analytic CIL via a […]

Ver mais

[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta). A model that can answer any question with regard to factual knowledge can lead to many useful and practical applications, such as working as a chatbot or an AI assistant🤖. In this post, we will review several common approaches for building such an open-domain question answering system.

Ver mais

Neural network weights are typically viewed as the end product of training, while most deep learning research focuses on data, features, and architectures. However, recent advances show that the set of all possible weight values (weight space) itself contains rich structure: pretrained models form organized distributions, exhibit symmetries, and can be embedded, compared, or even generated. Understanding such structures has tremendous impact on how neural networks are analyzed and compared, and on how knowledge is transferred across models, […]

Ver mais

arXiv:2602.16722v1 Announce Type: new Abstract: This paper introduces a real-time method for reverse engineering a vehicle’s CAN bus without prior knowledge of the vehicle or its CAN system. By comparing inertial measurement and CAN data during significant vehicle events, the method accurately identified the CAN channels associated with the accelerator pedal, brake pedal, and steering wheel. Utilizing an IMU, CAN module, and event-driven software architecture, the system was validated using prerecorded serialized data from previous studies. This data, […]

Ver mais

Because I write about LLMs (and maybe because of my em dash text replacement code) a lot of people assume that the writing on my blog is partially or fully created by those LLMs. My current policy on this is that if text expresses opinions or has “I” pronouns attached to it then it’s written by me. I don’t let LLMs speak for me in this way. I’ll let an LLM update code documentation or even write a […]

Ver mais