Working on the new implementation.

Hi!

So after coming back from the conference I had a plan in mind of how things were to be executed. 

The two main tasks were - 

1. Refactoring the dkimpy API for adding ARC support.

2. Developing a standalone ARC module for integration into Mailman.

(Here package refers to the `dkimpy` package)

Since the package was a dependency for the ARC module, it was necessary to proceed in order. After some thinking I decided to add an Enum class to the package with the different types of possible signatures. This could then be passed as an argument to the pre-existent signing/verifying methods. It was made a parameter with the default argument of the signature type DKIM. This was necessary since adding a non-default argument would break all the existing tests and function calls. Also, we wanted to keep the API change minimal, so as to not break setups after the intended upstream merge. 

Now, the ARC set of fields and the DKIM field vary slightly in terms of which "tag - value" pairs are included. However the underlying construction remains the same in the general view.

So now based on the signature type parameter which could be either - dkim, ams or aseal, different tags were added to the signature, using conditional statements.

One of the hurdles I faced here was verifying the ARC Seal. The ARC Seal does not specify the set of headers it hashes in the header hash tag, so it has to be constructed on the receiver side from the ARC Seal instance value. From my (flawed :/) understanding before the signing/verifying functions, given the list of headers, would sign/verify them bottom up which made me construct the headers so -

for i in range(1, idx):

headers.append(b'arc-seal')

headers.append(b'arc-message-signature')

headers.append(b'arc-authentication-results')

headers.append(b'arc-message-signature')

headers.append(b'arc-authentication-results')

However, the draft clearly mentioned the format with the places of the `arc-authentication-results`

and `arc-seal` exchanged in the above logic. I was aware of this, but had tweaked the order as according

to my understanding the signing was bottom up, so I was right. This was one problem that I had no clue

how to approach!

After spending quite some time thinking, an idea finally occurred! As a temporary modification, I added

the header "tag" to the ARC Seal and manually inspected the signature generated to see if the order was

right. And voila, I was wrong :P

The correct order -

for i in range(1, idx): headers.append(b'arc-authentication-results') headers.append(b'arc-message-signature') headers.append(b'arc-seal') headers.append(b'arc-authentication-results') headers.append(b'arc-message-signature')

I changed the order and the entire test message data (in the ARC module, point 2)

had to be changed too since they had implicitly used the older signing.

With that the package refactoring was completed. Yay!

Working on the second package was fairly straightforward. I had to write scripts which would

parse the message and make the calls to the package for the verification and subsequent

signing of the messages.

The only tricky area was to correctly verify the ARC Seal. The exact algorithm is outlined in the

draft but it still wasn't a straightforward implementation and took some time.

Tests were added and the final pieces were filled in like the setup script and a simple Python script

that processes the message through the entire pipeline and provides the results on the terminal for

user's convenience.

And that completed the ARC module for Mailman!

Now, currently there are two things scheduled in the pipeline -

1. Sending our refactored dkimpy package MR to the upstream original author for his review and

possible merge.

2. Participating in the next interop for live testing of our package in mail exchanges over

the internet with other similar implementations.

I'll keep updating the blog the details for the above.

Thanks for reading!

Aditya