README.md

---
title: Basic Smol Talk Chatbot
emoji: 🐠
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 5.15.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: A replica of the basic smoltalk chatbot I run locally
---

# Basic SmolLM2 chatbot:

This is a very basic chatbot using HuggingFaceTB/SmolLM2-[x]-Instruct hosted on the same host as the app. It is basically a replica of the chatbot I have for running locally.

## Hardware Scale Up

- I recommend running this on a little better hardware than this is set up with.
- I'm using the free tier space, but it needs a few more CPU to write fast enough to be useful.
- It is rather slow on this setup, but when I run this on my laptop, it works very well on CPU without GPU.

## To run locally

- Download the files or clone the repo.
- Make sure you have a supported version of transformers and torch installed (or run `pip3 install -r requirements.txt` from the root folder of this repo).
- Run `python3 app.py` from the root folder of this repo.
- Set your browser to http://0.0.0.0:7860


## Configuration options

- In app.py, there are settings  of the screen.
    - If you run this locally on a laptop with at least 5CPU cores, I would recommend saving all your local work then set `MODEL` to HuggingFaceTB/SmolLM2-360M-Instruct.
    - If this works without signs of resource saturation, try setting `MODEL` it to HuggingFaceTB/SmolLM2-1.7B-Instruct. This will write well and works fine on my laptop that is about 2 years old.

```
MAX_NEW_TOKENS = 250
MODEL="HuggingFaceTB/SmolLM2-135M-Instruct"
# MODEL="HuggingFaceTB/SmolLM2-360M-Instruct"
# MODEL="HuggingFaceTB/SmolLM2-1.7B-Instruct"
TEMPERATURE = 0.6
TOP_P = 0.95
REPETITION_PENALTY = 1.2
```