I am using openAI Assistant, I get the streamingUpdate using

streamingUpdate = assistantClient.CreateRunStreaming(thread, assistant);

I could iterate through streamingUpdate and send the response to the client using SignalR api. however I would like to know how to count the #of tokens consumed in the streaming mode.

I am using openAI Assistant, I get the streamingUpdate using

streamingUpdate = assistantClient.CreateRunStreaming(thread, assistant);

I could iterate through streamingUpdate and send the response to the client using SignalR api. however I would like to know how to count the #of tokens consumed in the streaming mode.

• c#
• artificial-intelligence
• openai-api
• azure-openai

Share Improve this question Follow edited Jan 8 at 19:39 General Grievance 5,0433838 gold badges3737 silver badges5656 bronze badges asked Nov 18, 2024 at 10:14 Sankaranarayanan RajkumarSankaranarayanan Rajkumar 5111 silver badge77 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

I found the way to get the token count

CompletionTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.CompletionTokens).SingleOrDefault();
PromptTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.PromptTokens).SingleOrDefault();
TotalTokens = assistantClient.GetRuns(threadID).Select(f => f.Usage.TotalTokens).SingleOrDefault();