Davin Studer

In my previous post Remove Jira Issue Attachments by MD5 Hash I showed how to remove attachments from JIRA based on the MD5 hash of the attachment.

I was feeling pretty good after writing that post and having eaten my doughnut. So, I went to tell a couple of my colleagues about it. This was their reaction …

So, you expect me to …

1. know what an MD5 hash is?
2. know how to get the MD5 hash of a file?
3. know where to find this script to add the hash to?
4. not mess the whole thing up in the process?

Um … uhh … yes? Ok, so maybe my approach isn’t super easy except to the programmer type. And now that I think about it I don’t want to have to be the one to always fix these. So, back to the drawing board. Let’s get this right.

So, I need to make it easy for others than myself to help maintain. Maybe if I made a way for my colleagues to take an attachment from an issue ticket and simply drop to a centralized storage location that could be scanned by the script … yeah that could work. It involves no knowledge of MD5 hashes or scripting and should be easy for pretty much anyone to do.

Now if I only had a location where we could place these attachments. A place that JIRA is able to scan. A place that all my colleagues have easy access to. If only such a place actually existed … hmm … oh, wait!! I could just have them attach the files to another JIRA ticket that will be used as a control ticket of sorts. Any attachments attached to this ticket would be compared against by the script and if a match is found then the issue attachment is deleted. (insert Handel’s Messiah playing in my head here)

The great thing is that most of my script doesn’t really need to be changed. All I need to do is specify a control ticket key in the script and have the script build the list of hashes based on that ticket. Here is my ticket …

And here is the new script. I’ve cleaned it up a little from the last version and removed a call to a method that is currently set as deprecated. It still worked even with the call, but best to get rid of that call before Atlassian removes the method altogether. Simply replace “{Project Key}-{Issue Number}” on line 12 with the issue key that holds your attachments to remove. So, if for instance the issue is in the FOO project and the issue number is 789 then that line would look like this …

def controlIssue = “FOO-789”;

import com.atlassian.jira.component.ComponentAccessor;
import com.atlassian.jira.issue.AttachmentManager;
import com.atlassian.jira.issue.attachment.FileSystemAttachmentDirectoryAccessor
import com.atlassian.jira.issue.Issue;
import com.atlassian.jira.issue.IssueManager;
import java.security.*;

/***********************************************************************************/
/* This is the ticket that has the attachments on it to compare MD5 hashes against */
/***********************************************************************************/

def controlIssue = "{Project Key}-{Issue Number}";

/***********************************************************************************/
/*                                                                                 */
/***********************************************************************************/

/************************************************************/
/* Don't edit below this unless you know what you are doing */
/************************************************************/

// Get the attachment hashes for our control issue to compare against
def attachmentHashes = getAttachmentHashesFromIssue(controlIssue);

// Obviously we don't want to run this on the control issue ... only on other issues.
if(event.issue.key != controlIssue) {
    deleteMatchingAttachments(attachmentHashes);
}

public void deleteMatchingAttachments(List<String> deleteHashes){
    def issue = event.issue;
    def attachmentManager = ComponentAccessor.getComponent(AttachmentManager);
    def attachments = issue.getAttachments();
    def attachmentFile = null;
    def bytes = null;
    def md = MessageDigest.getInstance("MD5");
    def digest = null;
    def hash = "";

    // Loop through each attachment on the issue
    for(a in attachments) {
        attachmentFile = getAttatchmentFile(issue, a.getId());
        bytes = getBytesFromFile(attachmentFile);
        digest = md.digest(bytes);
        hash = String.format("%032x", new BigInteger(1, digest));

        // Compare hash to the list of hashes we don't want
        for(h in deleteHashes) {
            if(hash == h) {
                attachmentManager.deleteAttachment(a);
                break;
            }
        }
    }
}

public List<String> getAttachmentHashesFromIssue(String controlIssueKey) {
    def deleteHashes = [];
    def attachmentManager = ComponentAccessor.getComponent(AttachmentManager);
    def issueManager = ComponentAccessor.getComponent(IssueManager);
    def issue = issueManager.getIssueObject(controlIssueKey);
    def controlIssueAttachments = attachmentManager.getAttachments(issue);
    def attachmentFile = null;
    def bytes = null;
    def md = MessageDigest.getInstance("MD5");
    def digest = null;
    def hash = "";

    // Get hashes for all the attachments in the control issue
    for(a in controlIssueAttachments) {
        attachmentFile = getAttatchmentFile(issue, a.getId());
        bytes = getBytesFromFile(attachmentFile);
        digest = md.digest(bytes);
        hash = String.format("%032x", new BigInteger(1, digest));

        deleteHashes.add(hash);
    }

    return deleteHashes;
}

public byte[] getBytesFromFile(File file) throws IOException {        
    def length = file.length();

    if (length > Integer.MAX_VALUE) {
        throw new IOException("File is too large!");
    }

    def bytes = new byte[(int)length];

    def offset = 0;
    def numRead = 0;

    def is = new FileInputStream(file);
    try {
        while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
            offset += numRead;
        }
    } finally {
        is.close();
    }

    if (offset < bytes.length) {
        throw new IOException("Could not completely read file " + file.getName());
    }

    return bytes;
}

public File getAttatchmentFile(Issue issue, Long attatchmentId){
    return ComponentAccessor.getComponent(FileSystemAttachmentDirectoryAccessor.class).getAttachmentDirectory(issue).listFiles().find({
        File it->
         it.getName().equals(attatchmentId.toString())
    });
}

And now my colleagues sing my praises (in my dreams) instead of cursing my name (which maybe still happens when I make hard to update workflows). Oh well, you live and learn.

Recently, we started using Jira at work to track some IT related things. Thus, I have quickly had to learn how to administer Jira. One thing that I really wanted to get set up and working well was the ability to respond to an email sent by the system via a reply email and have that filed in the ticket. That wasn’t too terribly hard to set up. First I created a mailbox for Jira to check in our mail system. Then I set up a mail handler to pull the reply emails in. I settled for using a “Add a comment before a specified marker or separator in the email body” handler so that I could provide a regular expression to define how to extract just the reply. The below screenshots show my setup for this.

The Split Regex field below is this /[Ff]{1}rom:[^\n]+{myuser}@{mydomain}\.{extention}/. Replace the {myuser}, {mydomain}, and {extention} parts with the email address of the account that Jira mails as. So, if your Jira system sends email as jira@coolstuff.org the above expressions would look like this /[Ff]{1}rom:[^\n]+jira@coolstuff\.org/. This will split the email where it sees the first line that looks something like this From: Jira [mailto:jira@coolstuff.org] or from: Jira [mailto:jira@coolstuff.org] … which is how Outlook formats its replies.

So, I got that part working great now I can go get a doughnut right? Nope! Turns out every time I would reply the issue icon and the image attachment in my email signature would get attached to the issue … over and over and over. So, before long I had a veritable glut of the same images attached to the issue. Grrrr!! So, I asked myself “Self, what can we do about this?” To which I so helpfully replied to myself, “Go check the Atlassian Marketplace, Atlassian Community, and Google for an answer.” After a couple grueling hours of trying to find the answer I came to the stark conclusion that there wasn’t one. Double grrrr!!

After a bit of thinking I decided I could just scan the images against a set of MD5 hashes to exclude when the issue is updated and here is the fruit of my labor. This solution requires having ScriptRunner for Jira installed. If you don’t have it … well, you should. The possibilities are pretty endless with what you can do with it. I created a Script Listener that would respond to the “Issue Updated” event.

And the actual contents of the script file.

import java.security.*;
import com.atlassian.jira.issue.Issue;
import com.atlassian.jira.component.ComponentAccessor;
import com.atlassian.jira.issue.AttachmentManager;
import com.atlassian.jira.util.AttachmentUtils;

// Add new MD5 hashes to the below array to auto remove them when they are attached to the ticket.
// This is helpful to get rid of things like images in email signatures, JIRA issue type icons in the email, etc.
def deleteHashes = [
    "eaf938ae5025889b60029d6d839d19db", //JIRA blue check mark
    "f370264d9a3d1b92666419e6ecc102ef", //email signature logo v1
    "662b051e6082e4499079ddc18e5eb302", //email signature logo v2
    "a4ab3c522859297084064502477effd8"  //Pulse line icon
];

def issue = event.getIssue();
def attachments = issue.getAttachments();
def attachmentFile = null;
def bytes = null;
def md = MessageDigest.getInstance("MD5");
def digest = null;
def hash = "";
def manager = ComponentAccessor.getComponent(AttachmentManager)

for(a in attachments) {
    attachmentFile = AttachmentUtils.getAttachmentFile(a);
    bytes = getBytesFromFile(attachmentFile);
    digest = md.digest(bytes);
    hash = String.format("%032x", new BigInteger(1, digest));

    for(h in deleteHashes) {
        if(hash == h) {
            manager.deleteAttachment(a);
            break;
        }
    }
}

public byte[] getBytesFromFile(File file) throws IOException {        
    // Get the size of the file
    long length = file.length();

    // You cannot create an array using a long type.
    // It needs to be an int type.
    // Before converting to an int type, check
    // to ensure that file is not larger than Integer.MAX_VALUE.
    if (length > Integer.MAX_VALUE) {
        log.info("File is too large!");

        // File is too large
        throw new IOException("File is too large!");
    }

    // Create the byte array to hold the data
    byte[] bytes = new byte[(int)length];

    // Read in the bytes
    int offset = 0;
    int numRead = 0;

    InputStream is = new FileInputStream(file);
    try {
        while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
            offset += numRead;
        }
    } finally {
        is.close();
    }

    // Ensure all the bytes have been read in
    if (offset < bytes.length) {
        log.info("Could not completely read file " + file.getName());
        throw new IOException("Could not completely read file " + file.getName());
    }

    return bytes;
}

Now, when I respond to an issue via email if any of the attachments on the issue match any of the MD5 hashes at the top of the script that attachment will get deleted from the issue. And if I find that there are other attachments we start seeing like this on a regular basis all I have to do is add the MD5 hash to the list and save the script … problem solved.

Now, about that doughnut.

Update: So, the whole editing the script and having others put in MD5 hashes part … yeah, that went over like a lead balloon. Here is an updated version that is much easier to administer.