August 2025

I’m not the only one who’s noticed that some people, even journalists, will ask chatbots like ChatGPT why they did something, and then treat the chatbot’s explanation as if it means anything. Or they’ll ask the chatbot to generate an apology, and then treat its apology as if the chatbot is really reflecting on something it did in the past, and will change its behavior in the future. ChatGPT is great at generating apologies. ChatGPT, of course, made […]

Ver mais

TL;DR Prompt injection, a security vulnerability in LLMs like ChatGPT, allows attackers to bypass ethical safeguards and generate harmful outputs. It can take forms like direct attacks (e.g., jailbreaks, adversarial suffixes) or indirect attacks (e.g., hidden prompts in external data). Defending against prompt injections involves prevention-based measures like paraphrasing, retokenization, delimiters, and instructional safeguards. However, detection-based strategies include perplexity checks, response analysis, and known answer validation. Some advanced tools also exist such as prompt hardening, regex filters, and […]

Ver mais

Three challenges in machine-based reasoning Translating from natural to structured language, defining truth, and definitive reasoning remain topics of central concern in automated reasoning, but Amazon Web Services new Automated Reasoning checks help address all of them. Automated reasoning Byron Cook August 04, 02:02 PM August 06, 10:17 AM Generative AI has made the past few years the most exhilarating time in my 30+-year career in the space of mechanized reasoning. Why? Because the computer industry and even […]

Ver mais

In recent years, Large Language Models (LLMs) have mostly improved by scaling. This has primarily involved increasing the size of the LLMs and the data they are trained on, resulting in a highly resource-intensive process that can cost up to millions of dollars. While LLMs have become ubiquitous, the resource-intensive pre-training process poses a threat to the inclusion of low-resource languages, where data is scarce. Often, this is accompanied by a lack of funding for compute resources. In […]

Ver mais