Spaces:

mebubo
/

gpted

Sleeping

mebubo commited on Mar 4

Commit

2d12d3d

1 Parent(s): cefa0ab

Chunk size 16 -> 64

Files changed (1) hide show

expand_llm.py CHANGED Viewed

@@ -39,7 +39,7 @@ class LLMBatchExpander(BatchExpander):
     model: PreTrainedModel
     tokenizer: Tokenizer
     threshold: float
-    chunk_size: int = 16  # Default chunk size, can be adjusted as needed
     def expand(self, batch: Batch) -> BatchCandidates:
         start_time = time.time()

     model: PreTrainedModel
     tokenizer: Tokenizer
     threshold: float
+    chunk_size: int = 64
     def expand(self, batch: Batch) -> BatchCandidates:
         start_time = time.time()