Coder-Vippro/ChatGPT-Discord-Bot
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases 13 Wiki Activity

Improve history trimming logic to remove oldest messages first for better token management

Browse Source
This commit is contained in:
cauvang32
2024-12-30 18:31:06 +07:00
parent 6bd29626f9
commit 36c94d64a6
2 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
bot.py
View File

@@ -615,11 +615,12 @@ async def handle_user_message(message: discord.Message):
# Function to trim the history to avoid exceeding token limits
def trim_history(history):
"""Trims the history to avoid exceeding token limits."""
"""Trims the history to avoid exceeding token limits by removing older messages first."""
tokens_used = sum(len(str(item['content'])) for item in history)
max_tokens_allowed = 9000
while tokens_used > max_tokens_allowed:
removed_item = history.pop(1)
# Remove from the front (oldest) while total tokens exceed limit
while tokens_used > max_tokens_allowed and len(history) > 1:
removed_item = history.pop(0)
tokens_used -= len(str(removed_item['content']))
# Function to send a response to the channel