I only transformed the normals when I did rotations, thus I kept a separate rotation matrix that had all of the rotations. I applied the rotations to the vertices as well as to the normals. However, the normals did not receive translations or scaling operations.
I took swipes to the left and right to be a call to rotate in the y direction (the amount swiped is proportional to the angle). A tap on the left or right of the screen translates in the x directions. The tap on bottom or top of the screen is translate in the z direction. I used swipes up and down to translate in the y direction. Lastly a double tap in the center of the screen scaled it up by 1.5 and a triple tap scaled it down by .5.
