Anderson Status Update: the Obsessive Quest for Codebooks and OCR Fun

 I truly love my thesis topic and am so happy I chose it. However, it is causing great difficulties as I avoid all my other work because I am having so much fun. I have completely messed up my sleep schedule because I get home, start checking OCR’s and looking for codebooks, then suddenly it’s 3:00 AM. I have been searching for every code book under the sun, and have found at least 10 books that were not used by NARA. Also, I currently theorize that NARA’s focus on obtaining codebooks, rather than the changes in punch card regulations, was a major contributor to their decoding errors. I am using ABBYY fine reader to use OCR on the codebooks included in NARA’s documentation. This is taking far longer than expected, mainly due to poor scan qua, however it is also incredibly fun. The challenge is that it is very difficult to stop because I want to keep going through the books. Thus far I have extracted about 200 pages - the counties and states for 1944 (3,095 rows) and station codes for 1945 (I have not finished this yet but I estimate 2,000 rows). 

Additionally, the Library of Congress holds a complete copy of the 1942 and 1943 codebooks, which would be highly useful to see, but they are each around 500 pgs and not currently digitized. I am trying to determine a cost effective way to obtain copies. 

So far I have learned quite a lot about military manuals and the Adjunct Generals Office. HathiTrust has proved the most useful. The University of California hosts the largest collection of Technical Manuals I can find (of course they are missing the main one I need). I have also learned that despite being 80 years old, you must have a CAC to access the TM ‘s from the 1940’s if you go through the Army publication site. They also have some TM s marked as active from the 1930s - however they are for series 9 (Ordinance Dept) not series 12 (AGO). For context, I am looking at TM 12-310 and 12-305. I think the AR reports should be looked into for coding processes, however (much to my dismay) there are so many possible numbers to look at it is not feasible for this project. 

As I write this, I am coming to terms with the fact that I must stop seeking out additional codebooks due to time. However, I am still determined to obtain the LoC TM 12-310 for personal interest. If I am able to, I will keep a version on my computer, make copies of at least two dedicated hard drives to put away for safe keeping in separate locations, and print a copy, for which I will order a perfectly sized box from Gaylord. I have looked at every website, database, catalog, and retailer I can find that has anything to do with WWII, the Army, and technical manuals, but I have not been able to find a complete copy - there isn’t even a forged insanely overpriced copy on EBay or AbeBooks. 

The ‘actual’ challenge has been the literature review. This is not due to lack of material, rather that my sources struggle to communicate. I am starting from the beginning. As there is always the possibility that I missed something, am using the wrong search terms, or am looking in the wrong places, I hesitate to say there is no research on processing intrinsic big data or extracting the spatial aspects of said data. What I can say is following hours upon hours of research, I can’t find anything on the topic. So, I found a wide variety of data handling practices from archeology, machine learning, and cultural heritage. Essentially, I am looking at these sources, and picking aspects of each to develop a workflow. Perhaps this is paradoxical as by creating a workflow from multiple sources I am creating a commonality and thus they do communicate. However, this does make their communication quite difficult.

I decided to add a “Data Review” as I realized in many ways the most critical part of this project is understanding the data I am working with. This may sound simple but the data is in no way clean cut. In fact it’s a complete mess and everything conflicts. The more I learn about the data, my shock that the enlistment file database is available for public use increases exponentially. 

I do think this again raises a question which I continue to mull over. The enlistment file is one of the most used, most critical, and most relevant archival records set for genealogists. Yet every time you search for a record, there is at least a 74% likelihood that a record you are looking at is inaccurate. Websites such as ancestry.com pull data from AAD, increasing the output of misinformation. However, is it better to have your great grandfather's rank spelled “p0t” instead of “pvt” or not have it online at all? 

Personally, I feel like it should remain available despite its inaccuracy, however it needs to be plastered all over the website that the records contain a high degree of inaccuracy, and if you don’t find someone, it doesn’t mean they aren’t there or the information is correct. 

Overall, I think the most remarkable aspect is that there is no active effort whatsoever to reprocess the records. In some ways this research is pointless, because without accurate images there is no way to check digitization accuracy. The microfilm needs to be reprocessed and reanalyzed - it’s been 25 years since anyone used the original microfilm. However perhaps the point of this research is to show what could be possible if the records were reimaged. 

First Update

Since my introductory post, I have honed in on a specific project which I will use to expand my experience in archives and hopefully contribute to ongoing conversations about archival principles of arrangement and description. I will be working on a collection housed at the Rochester Public Library of the personal records of Dr. Lewis Lansky, a former MCC History professor and member of the civil rights action group F.I.G.H.T. (Freedom, Integration, God, Honor, Today), and the later group Friends of F.I.G.H.T., established for non-black allies in Rochester. 

Through some initial research I have found an oral history interview with Dr. Lansky, along with two articles he wrote for the MCC student paper on his first-hand experiences marching with Dr. King and working with other civil rights leaders.

I am so excited to begin working on this project, which will involve fully processing it and will culminate in a finding aid created through ArchivesSpace. 

Update: From Living History to Visual Culture Theory

 Due to sickness this past Monday, my first project update arrives a week later than my peers. I am actually glad that it worked out this way as prior to the submission of the first draft of the lit review I likely would have focused the uncertainty in my ability to actually complete that assignment, having never done one before. In the wake of our first submission, I am reflecting on some gaps in my project that are included in my thesis but not supported by the exhibition at present. For example, an exhibition in the University Gallery is advantageous for numerous reasons. The fact that it's a white cube type of infrastructure lends itself well to the problem I address through my criticism of Modernist curatorial methodology. Additionally, it also functions as an event space so audiences with varying levels of investment in and understanding of art exhibitions will experience my subversive method of curatorial praxis. However, I do not have access to the institutional documentation that supports that gallery space and it does not have an interpretive development policy in the way that art museums ideally would. In this way, the gallery space is almost a transplant environment and my exhibit analogous to a traveling show. Communal response should directly influence RIT Archives institutional management approaches. This presents a unique challenge as visitors are disassociated from the Archives central collection and exhibit spaces when interacting with my exhibit. I'm currently mulling over a tagging interactive that could be utilized by the art team once the exhibit comes down.

The most significant change my project has underwent since I last posted is that I am no longer primarily pulling from living history and am instead am focusing on different visual culture theorists like Eileen Hooper-Greenhill, Grant Kester, Amelia Jones, and Tracey Warr. Feminist performance art remains one of the primary arbidges between passive and active viewership, and for the most part the project has stayed fundamentally the same. I'm looking forward to further developing curatorial text for my exhibition in the coming weeks.

First Update: Danni Kuntz

 Hello again!

I feel as if a lot has happened since my first post. Firstly, and most importantly, I chose a specific topic: Everyday Life in Ancient Rome. I think this project has a lot of potential, and it is helping me prepare for my future plans. In the time since my last post, I've compiled my sources and annotated them, written an abstract, and even have started to work on a literature review for my project. It's been a lot of work, but I've also had a lot of fun delving into my topic of choice. 

Capstone Update 10/01/2026

Since last blog post, I have changed the goals and output of my capstone. I decided to go from a paper to a project because it will allow me to gain tangible experience in operating as a museum professional. The project will still be centered around LGBTQIA2S+ representation in a GLAM setting, like my proposal originally was. The main difference is what my thesis has changed into. Now I plan on gathering an oral history interview with a steward of LGBTQIA2S+ resources within a library setting. This interview will be the basis of my capstone, complimented by research on both the standards of conducting the oral history interviews of queer communities and on the operations of an LGBTQIA2S+ library.

Because of this change of plans, I am not quite as on track as I had hoped to be at this point in our fall semester. My first order of business since this reorientation is to find all of my sources, annotate them by next class and work on a full draft of a literary review. I also need to identify a subject for the project and receive their permission for an interview. My methods are not yet determined for the interview portion, and I do not have any present experience with oral history, but I hope to consult with Museum Studies and Women's, Gender, and Sexuality Studies staff members to come up with a solid plan on how to approach the session. 

-Bella

First Update

Since my last blog post, I have further honed my idea and found more sources that I have annotated. I will still be comparing and contrasting the films Barry Lyndon and Andrei Rublev to examine how they represent and interpret the past differently and make sense of fragmented history. In addition to comparing and analyzing the films, I will be relating how the creation and interpretation of history in these films relate to curation and interpretation in the museum world. Ultimately, I hope to showcase the strength of these films and to make the case that film/art can play a role in understanding the past. 

Changing Directions + Status Update

Hello! 

Since my initial post, the direction of my project has shifted considerably. I am still doing research regarding digital collections, but my new research question is "How can multispectral data and IIIF annotations be used to enhance user interaction, discovery, and scholarship in digital collections?" Multispectral imaging has the potential to completely reframe collection items, but results are generally publicly unavailable or stored in separate repositories. IIIF allows image stacks to be created and manipulated in digital collections through its choice cookbook recipe, but this feature remains underutilized in the GLAM field. I hope to create a small prototype digital collection using medieval manuscript leaves to showcase this functionality. One of my goals with this collection is to ensure compatibility with manuscript reconstruction projects like Fragmentarium, since having these images present with their true-color counterparts could reveal new insights into these codices.

So far I have written my abstract and made significant revisions to it. I have identified around 20 sources that I will use for my project. One of them has significantly influenced how I think about my project: Hannekan's 2023 article that introduces the concept of "deep digitization"-- combining multiple imaging modalities in an online environment that allows objects to be studied in the broadest way possible. Currently, I am finishing my annotated bibliography and starting my lit review. I'm trying to figure out how I want to structure it, which I anticipate will probably be the most difficult part. I also have gone down a few rabbit holes related to (but not really contributing to) my project. Otto F. Ege's "I am a Biblioclast" is such a fascinating glimpse into the method behind his madness. I can't stop thinking about how his eight year old daughter could not sleep until an illuminated manuscript leaf was hung in her room. 

All the best, 

Alec 

Update on WWII Enlistment Records

 So far, my thesis is going well. The most difficult part is holding myself back from beginning data processing. I’ve scoured the internet and found that, as I suspected, there is little to no research on standard data management for internally derived datasets in cultural heritage. This works in my favor, as I can use this project to fill that gap.

My plan is to take GIS data ethics from archaeology and data management strategies from machine learning, weave them together, and create a framework to serve the needs of cultural heritage institutions. On that note, I need to define a word for cultural heritage that excludes archaeology. I find that there is a conflation between archaeological GIS and historical GIS. Actually, I would go so far as to say there is a difference between historical GIS and the point of this project. In archaeological GIS the spatial data comes from the object's location. In historical GIS there is both intrinsic and extrinsic spatial data. This generally takes one of two forms: A map (intrinsic) or provenance (extrinsic). What this project works with is a collection with intrinsic textual spatial data. That is to say there is no spatial data within the object or about the object, we are interpreting the text to obtain spatial data. Technically, this is intrinsic data, however it is clearly different from a map.

Returning to the point, I have been messing around with the 2002 WWII enlistment data. I converted the ASCII code to hexadecimal, which with the help of Claude was quite easy, but did increase the file size to 5 gb (yes gb). I was expecting 50-60 gb. Now that the data is in hex, it needs to be decoded. To do so I will need to (a) figure out a way to extract all the codes from the documentation quickly and (b) how to effectively chunk the data to avoid sitting for hours while the code runs. Outside of the lit review the next step is figuring out OCR.