Answer
$\lambda=6.34*10^{-7}m$
Work Step by Step
The key to solving this question is realizing what the limiting factor for observing a 5th order, or any order for that matter, is. In a diffraction grating system, the maxima-fringe pattern is observed on a screen at some distance away from the grating. No matter what that distance is, the position of the maxima on the screen can be described using the angle $\theta$, which is the angle (shown in the attached image) between the diffraction grating, central maxima, and the chosen maxima on the screen. Hence, any fringe that is formed by the screen has to have an angle less than 90 degrees, as after that, the diffraction pattern leaves the screen.
Thus, the condition for maximal fringe formation on the screen is that $\theta<90$ degrees.
In mathematical form, we establish the equation
$$dsin\theta=m\lambda$$
where $d$ is the grating spacing, $\theta$ is the angle between the zeroth order maxima and the maxima fringe in question, $m$ is the order of the fringe and $\lambda$ is the wavelength of incident light on the diffraction grating.
Hence, to know the maximum wavelength that can be incident, for a 5th order maxima to be noticed, we set $m=5$, and $\theta=90$ degrees (refer to discussion about maximal fringe observable).
We are told that there are 315 grating rulings per millimeter. Hence the spacing between each grading is $\frac{1*10^{-3}m}{315}=3.17*10^{-6}m=d$
Hence, we can now plug that in the diffraction grating equation:
$3.17*10^{-6}m*sin(90)=5\lambda$
Solving for $\lambda$ we get $\lambda=6.34*10^{-7}m$
