human readable date formatter

[rlskoeser]

ideally should support conversion in both directions; implementation should support localization for so it works in multiple language; this either overlaps with or can leverage #78

• Create an initial prompt with EDTF context and few-shot examples as "training data"
• Create test data. Include multilingual examples (but not different calendars). Europeana? @rlskoeser will check with Pietr (missing data projects)
• Write a script or notebook to allow testing across multiple LLMs
• Get benchmarks for multiple LLMs; consider iterating across multiple versions of the prompt; think about how to cache the prompt, if possible
• If successful, turn this into a formatter (converter). Keep it (and any Python dependencies needed to run it) as optional.

[ColeDCrawford]

Cole and Rebecca tried out few-shot prompting for going from human-readable dates to EDTF. We provided the EDTF spec and a list of example string-EDTF pairs.

Cole tested GPT-4o. It did fairly well and responded well to corrections when it missed a kind of formatting - it parsed similar examples correctly after the correction. Rebecca tested Claude Sonnet 3.5; it did similarly well with formatting, but struggled with calendar conversions. We're going to stick with parsing human-readable dates to EDTF for now.

The core prompt:

I am providing the spec for the Library of Congress EDTF (Extended Date Time Format). I am also providing a list of paired plaintext strings with their EDTF representations as few shot examples. I would like you to respond to future queries of plaintext with the assumption that they are dates, and to parse them into the EDTF format. Return results with the JSON structure:

{
"input": <input_text>,
"edtf": <parsed_edtf>
}

EDTF_Date Conversion Experiment.md

[ColeDCrawford]

We are going to try to standardize a prompt with the needed EDTF spec and examples as context to include with each query. We create some test data and a script that we can use with multiple LLMs (maybe using litellm or another similar library) so we can benchmark against different large and small foundational models. If the results are promising, we can extend this into functionality that can be used in the core undate library.

[rlskoeser]

I was using the version of Claude integrated with the Zed editor, which takes advantage of Anthropic's context caching. Here's the prompt I started with:

@fetch https://www.loc.gov/standards/datetime/
Based on the specification for the Extended Date Time Format (EDTF) and examples of human readable strings and EDTF representation, please parse other plain text strings into EDTF format into this json structure:

{'input': <input_text>, 'parsed': <parsed_edtf>, 'notes': <comments>}

Here are some examples:
...

I was curious how or if it would handle calendar conversion, so I tried a few examples from the Princeton Geniza Project; it did ok on some of them but others had different results (or different precision) than what we have in PGP.

Here's the full transcript of my experiments, minus the contents of the LOC EDTF spec which the @fetch directive inserts at the top.
claude-date-parsing-experiments.txt

[rlskoeser]

@ColeDCrawford I got the chance to chat with @statsmaths about the kinds of dates he had to wrangle to use the FSA/OWA Library of Congress collection.

One of the interesting things that wasn't on my radar previously is that LLMs should do better at taking advantage of context and we will want to test; e.g. one of the photos in the examples Taylor shared was listed as July of a particular year in the date field but it's evident from the title that it is July 4th. Probably other general context would be relevant, like hemisphere if you're using seasons, or US / Britain context to disambiguate month/year vs year/month.

I've started a google sheets spreadsheet in the shared drive for this project with provisional structure for starting to collect example dates from different projects for whenever we're able to circle back to this.

I've set it up so we can keep track of when we have example dates from specific projects or sources.