How can I add a diary question to a specific department?

Go to Settings > Diary questions. As a default you will see '(All departments)', click on this button and select the appropriate department.