Answer
$ a = 1 $ or $ a = 25$
$ b = 0 $ or $ b = 13$
Work Step by Step
Enciphering function of the shift cipher:
f(p) = (ap + b) mod 26
Corresponding deciphering function:
f-(p) = a(p - b) mod 26
Note: a is the inverse of a modulo 26.
We want the two functions to be the same:
(ap + b) mod 26 = a(p - b) mod 26
Using distributive property:
(ap + b) mod 26 = (ap - ab) mod 26
The constant a then also has to be equal to a:
a = a (mod 26)
Or equivalently a = aa = 1 (mod 26).
There are only two values that are each others inverse (modulo 26):
a = 1 or a = 25
The constant b then has to be either 0 or 13 if the two functions are the
same (result from exercise 10 when a = a = 1, similar for a = a = 25).
