Another “AI” post with bonus hallucinations!
Since 2018 I’ve trained undergraduates how to utilize the Schaefer Archive and to transcribe the data and validate the analyses. Alice Sharpless is the fastest person I know for this work with the best eye for the unique orthography of the archive, to say nothing of her prodigious eye for unique features of the coins themselves. During our grant we were only able to transcribe and validate some of the issues (those that are done are now all up on CRRO and in sitnam). It is labor intensive with no idealized work flow. I’m sure human error slips in given all this. And catching human error is hard. All the material is however open access and begs the question, now what?!
So can Claude speed the work? Absolutely. Does it need human judgement and expertise to interpret and tweak the results? Absolutely. At the end of this post you can find the briefing the should let you replicate my work. But I’m going to give my process a human narration.
First I uploaded four high resolution scans of a single Schaefer archive page. These correspond to the binder page with RRC 419/2. This is Binder 7, page 108 (images 0,1,2,&3) in the original manual numbering of the pages during the image capture process by Erin Richardson. In Archer the ANS archival viewer you may see them numbered as pages 131-134.
I selected it as a small issue I’d worked on previously on this blog as I thought this would make a good test. I began by testing to see if it could extract data from a single coin. These files were not downloaded from the public ANS website but are project back up copies of open access material, jpgs at over 50 MB each. There are even larger TIFFs for archival purposes.
This zip file contains 4 high resolution images. These images are all of the same scrapbook page but because material was overlapped on the original four images were required to capture all the data. The goal is to create a table with each coin as a separate row and in the row all the data relevant to that coin. begin by just extracting the image of the coin in the upmost right hand corner and the hand written data relevant to it. Then wait for next steps.
1st Prompt
This was v fast and remarkably accurate. So I proceeded to ask it to do the whole page cross referencing the four different images. And I gave it a bit more information to do so:
All the coins on this page are RRC 419/2. No need to transcribe this going forward. Green has precedence over other die identification. Thus this is obverse 8 and we ignore the other ID. AR2 is the source, the citation in that source is 251. two short lines refer to a die link to a near by coin. Using this information transcribe all coin into a single illustrated table. Look for notes on the page that may apply to more than one coin. transcribe those separately.
2nd Prompt
This took about an hour to run. And used a great deal of my session allowance, but not all of it. The results were impressive only tweaking was needed:
Adjust artifact to presume Roman numerals are the same as arabic numerals when labelling obverse dies
Separate out those coins marked Plated or Hybrid. ensure the the image of the coin shows both obverse and reverse and does not cut off any handwritten information about the specific coin.
D5 is the same as D6. Combine records.
Prompts 3-5, entered sequentially
This got me a workable table and spreadsheet and let me see things that both Erin and Dick missed. Claude put into the notes that ST.7 411 (C8 = D3) appears twice on the page in two different positions. Luckily Dick (and now I) saw the same dies at work even with two different photos mistaken for two different coins. I also realize that even with four exposures Erin didn’t capture ALL the data. Overlaps still hide the source of one coin (D5) and another coin has the obverse partly covered (D7). This isn’t huge because this is small issue and I can readily spot these coins on the trade databases (see links).
Claude created the names for the coins and an interactive map to let me jump to each record in the table:
Here’s a sample of the transcription capability. It is cautious about what it transcribes and is good about flagging ambiguity.
This is what the general notes look like when transcribed:
I was impressed that Claude knew to shift languages when looking for notes and also deduced plating when coin photo itself was not labeled:
I then asked it to give me a die map.
Produce a Bipartite die-link graph with crossing minimisation, and then create an artifact that shows the minimum number of anvils necessary to have no overlapping lines.
Prompt #6
I did not give it a briefing from other dies maps I’ve created. This means it did the work with no frame of reference. Probably a mistake on my part. Need to create a briefing that specifies my specific preferences for die mapping.
Still it isn’t bad, even if it does not suggest a die box instead of multiple anvils. You’ll see it assumes multiple anvils is the only solution to reconstructing the manufacturing process. Below I prompt it to consider die box depth.
And it gave me all the variations possible:
[Claude briefly slipped into using “dice” as plural of “die”. 🤣 English is hard.]
assume one anvil, how deep is the die box?
Prompt #7
The first person “I” is unnerving, but I think having it re write it without would be even weirder. And I tried asking and got my first “AI hallucination”
So that was fun! It would be possible to upload Alice’s guide to decoding sources and teach Claude the format for transcribing the data to match format needed for integration into Numishare software (the platform that runs CRRO). Though, I think one would have to first check the results in another interface like the one Claude created before manually telling it to proceed with export.
My concern is not my usage here, it is what happens when someone uses this type of tool for a field on which they have no expertise. I can readily see Claude’s errors and tweak and correct. I know Schaefer’s handwriting. I know the material. I know how to discern good from bad. What I don’t yet know is how to teach a student to replicate my ability to check and use Claude responsibly for this type of task.
This raises concern about skill loss. I’m good at reading archival hands. Not all, but most I encounter. This type of ability to read script has been ‘special’ and something that takes practice along with a innate talent. If we rely on the machines how will we be able to check the machines.
I think of it as how I read a Loeb. I read the translation until I come to something interesting or odd and then I check the Latin or the Greek v v carefully. Often in my publications I tweak the human translations to show my own reading. Sometimes I translate from scratch, but only as necessary. I need the ability to do that.
Maybe a better metaphor is arithmetic. We made our kids memorize their times tables and all the rest including long division. They have calculators. I have an excel spreadsheet to keep me from making computational errors. I prefer to write a formula than do the sums long hand. But I could.
I’m most excited to use LLMs to speed up my tasks rather than try to become something I’m not. I could ask it to write music, but I have no ear. How would I know if it was any good? What part of it would be me? It holds no interest for me.
