Create or update bag to include optional/custom tag files?

[rlskoeser]

I was looking at the code to try to see if it's possible to create a bag with custom tag files, or update an existing bag to add them. I didn't see any code that looked like it would let me do this, although it does look like there is code for validating and finding custom tag files referenced in a tag manifest. I thought it wouldn't hurt to ask if there's a way to do this that I missed.

[johnscancella]

Hi @rlskoeser Are you talking about adding files and their checksums to the tagmanifest-.txt file? If so it should automatically do that when you call save; all you would need to do is add the file to the bag root directory

So the code would be something like

bagit.make_bag(/foo/yourBagDir)
#Add your custom tag files here
bagit.save()

[rlskoeser]

@johnscancella Thanks, that sounds like what I need. I'll try it out.

[rlskoeser]

@johnscancella That didn't do what I thought you suggested it would, but I think I probably misunderstood and/or (more likely) didn't communicate my initial question clearly.

According to section 2.2.4 of the bagit spec, bags can contain other tag files:

A bag MAY contain other tag files that are not defined by this
specification. Implementations SHOULD ignore the content of any
unexpected tag files, except when they are listed in a tag manifest.
When unexpected tag files are listed in a tag manifest,
implementations MUST only treat the content of those tag files as
octet streams for the purpose of checksum verification.

I didn't know about this feature until recently myself, when I was asked to build a script to create bags with custom tag files for various types of metadata. (I think this bagit profile may be adapted from the DPN bag spec.)

Does python-bagit have any way to generate checksums for or otherwise manage these kinds of other/optional tag files, so they can be validated along with the other bag contents? Or do I need to include them as payload content to do that?

[johnscancella]

Hmm it should have. Here is some example code

import bagit
import os

"""
A script for testing creating and updating your own custom tag files
"""

#create a test directory for bagging
directory = "testBag"
if not os.path.exists(directory):
  os.makedirs(directory)

#create a test file to bag
test_file = open('testBag/hello.txt', 'w')
test_file.write("hello!")
test_file.close()

#bag the test directory
bag = bagit.make_bag(directory)

#create our own custom tag file
myfile = open('testBag/mycustomtag.txt', 'w')
myfile.write("hello this is my custom tag file")
myfile.close()

#update the bag with the custom tag file
bag.save()

save it as a file and run it python <WHATEVER YOU NAMED IT>.py
now if you open up testBag/tagmanifest-md5.txt you should see the line 9efc41d01b86ac78f567618377ca2f46 mycustomtag.txt

[rlskoeser]

Aha! That's very helpful. I can run your code and see that it finds the file and includes in the tag manifest. If I add another file in the top level directory, it gets included in the tag manifest. However, if I add a subdirectory with a file, that file does not get included in the tag manifest.

Now that I know better what I'm looking for, it looks like the logic for finding files to include in the tag manifest is here:
https://github.com/LibraryOfCongress/bagit-python/blob/master/bagit.py#L769
However, it only looks in the current directory and does not look for or recurse into any non-data directories that may be present.

So, I guess now the question is - should it?

[johnscancella]

According to https://tools.ietf.org/html/draft-kunze-bagit-13#section-2 it looks like having optional tag directories is valid, so I would say that is a bug.

[johnscancella]

This should be fixed in the latest code. And you will be able to install it using pip once we release 1.5.5