Inference Proxy

Setup

This proxy captures and stores traces from LLM API requests to your personal Hugging Face dataset.

1. Duplicate Space

First, duplicate this space to your account to set up your own instance.

2. Set Environment Variables

Configure these required environment variables in your space settings:

HF_ACCESS_TOKEN=your_huggingface_token

USER_NAME=your_huggingface_username

Example Usage

JavaScript

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "{{HOST_URL}}/fireworks-ai/inference/v1",
  apiKey: process.env.HF_API_KEY,
});

let out = "";

const stream = await client.chat.completions.create({
  model: "accounts/fireworks/models/deepseek-v3",
  messages: [
    {
      role: "user",
      content: "What is the capital of France?",
    },
  ],
  stream: true,
  max_tokens: 500,
});

for await (const chunk of stream) {
  if (chunk.choices && chunk.choices.length > 0) {
    const newContent = chunk.choices[0].delta.content;
    out += newContent;
    console.log(newContent);
  }  
}