HistSift - OAI json history parser

HistSift

 (I just like the ist-ift assonance, forgive me the dumb name, it's really more a parser than a sifter but whatever)

tl;dr: a program that takes history.json and extracts each contained conversation into a separate, easily readable .html file which are each separated into folders based on the system prompt they were generated with (and also lets you cull redundant conversations for cleanliness' sake)

Hi there!

I was into computers and tech from a young age and learned basic HTML written via PICO on an Apple IIGS several decades ago.. and that's basically where my coding journey ended. I never really learned to code, though I know my way around a lot of it, and went on to do other IT related things, so now that my storied history of coding is out of the way..

I made this over the course of a few nights after being a little annoyed at how fiddly it became to manage conversation histories and parse through them in a readable fashion when messing around with playground prompts and such. Not to mention having to worry about losing them over time and then having even less of a way to read them due to the ugly and verbose json format they are exported in. 

The program is as boring as the logo the logo is stupid and tacky, which I actually did by hand as a sort've tongue-in-cheek-I-know-it’s-bad-but-at-least-it's-not-AI type thing~

And, let's be honest, if you have a bunch of conversation histories with varied prompts and you want to keep and sort them for easy review, well.. we both know why you're here. :) And that's fine, because that's why I decided to share this! 

I found a few scripts online but none quite scratched all the itches, and many were limited in function and based on command line usage. As a result I decided to give a try and used the usual LLMs to fuddle my way through making a powershell script and then moved that into a GUI format that'd be more easily downloaded and run by others. 

All you need is this program and a folder with your .json histories you've downloaded. Some of it is clunky, some of the GUI windows are a little off kilter, and it’s not pretty.. but I kinda just got sick of fussing with it when it does what it's supposed to do well enough!

It's probably only really compatible with Windows 10/11 and I cannot promise it to be especially error-free, robust to system variables or other oddness, nor in any way compatible with other operating systems or environments.

All that said here's what it does:

Takes an input of a history.json file which it then parses through. Every conversation is parsed out into a separate, easily readable .html file that has formatting akin to old IM windows because that felt pretty intuitive. The conversations are grouped into folders based on the system message prompt at the beginning - only the first sentence of the prompt is considered to minimize accumulating tons of folders because of small adjustments to the prompt which are basically for the same prompt. As a result if you keep the first sentence of a prompt the same it will always group conversations together accordingly. I also added the option to change the font size and the color of the user and non-user text before each message for a little personalization.

Because some folks might prefer to have a separate folder for any difference in prompt I also made it so that it will still prompt before extracting into an existing folder, and also show you the first sentence, so that you can create separate folders for similar/same prompts from different histories if you so choose.

While running it will prompt you for an alias for each unique system prompt, which will be the name of the folder its conversations will be sorted into and also an alias that the program will check against for future sorting.

I was also annoyed at how many redundant conversations there were due to the way the histories are saved, often resulting in many repeats that are identical but shorter versions of other, more complete conversations. The other button at the start of running the program will sort through all the extracted HTML conversations in the subfolders of the given folder and then prune them. Any conversation which is *wholly* contained in another conversation will be removed, but any which branch off into different directions/responses will all be retained. Only shorter versions of more full conversations will be deleted (and you get prompted first, too!)

And that's it! I hope this is helpful to someone out there and, if you feel super appreciative and happy for the time it saved you or the function it provided, feel free to throw me a few bucks for the time. Or just download it for free, that’s fine, I don’t really care, whatever works for you. <3