Tag: open-source

Intermediate

How Far Will They Go? Red-Teaming Online Influence with Large Language Models

Daniel C. Ruiz, Anna Serbina, Ashwin Rao, Emilio Ferrara, Luca Luceri

This research exposes systematic political biases in open-source LLMs and shows how simple jailbreaks can expand their 'Overton Windows' — the range of political opinions they'll express. The findings reveal that most open-source models lean left, smaller models are more politically constrained, and recent models are often more biased than older ones. Essential reading if you're deploying LLMs in contexts where political neutrality matters or if you need to understand the security implications of model political expressivity.

Takeaways

Open-source LLMs show systematic left-leaning biases and can be easily jailbroken to express more extreme political positions.
Model size inversely correlates with political expressivity — smaller models are more constrained in their political range.
Simple natural-language jailbreaks can significantly expand the political opinions a model will express on controversial topics.

from Jun 1, 2026 · via api-hf · arXiv:2605.22880

Intermediate

Benchmarking Ollama vs LM Studio vs MLX

llms open-source

A hands-on performance comparison of three popular local LLM inference tools (Ollama, LM Studio, MLX) that investigates why one tool felt laggy in practice. If you're choosing between local inference options or debugging performance issues with self-hosted models, this benchmarking approach shows how to systematically evaluate tools beyond just theoretical specs.

Takeaways

Perceived performance issues with local LLM tools require systematic benchmarking beyond just checking specs on paper.
The three major local inference platforms (Ollama, LM Studio, MLX) have measurable differences that affect real-world usage.
Proper benchmarking methodology for LLM inference tools should account for both throughput and latency characteristics.

from Apr 27, 2026 · via suggestion

Intermediate

Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

ikessler

agents llms software-engineering open-source

This Chrome extension demonstrates practical browser-based AI deployment by embedding Google's Gemma 4 model locally via WebGPU, complete with webpage interaction capabilities like clicking, typing, and JavaScript execution. It proves that sophisticated AI agents can run entirely client-side without API dependencies, opening new possibilities for privacy-preserving AI tools. The implementation shows how to build truly local AI agents with real-world utility.

Takeaways

WebGPU enables running 2B parameter models entirely in the browser without cloud dependencies.
Local AI agents can interact with web pages through tool calling while preserving user privacy.
Browser-based AI deployment eliminates API costs and latency while maintaining reasonable functionality.

from Apr 6, 2026 · 100 points on HN · via api-hn

Intermediate

Streaming experts

llms open-source

Breakthrough technique allows running massive Mixture-of-Experts models (up to 1 trillion parameters) on consumer hardware by streaming only the necessary expert weights from SSD for each token. This could democratize access to state-of-the-art models for teams without enterprise-scale infrastructure, though with latency tradeoffs.

Takeaways

Streaming expert weights from SSD enables running models 10x larger than available RAM would normally allow.
The technique makes trillion-parameter models accessible on consumer hardware, potentially changing deployment economics.

from Mar 29, 2026 · via rss-willison