I developed one application for iPhone and now I want it on App Store. So many my iOS geek friends told me to test it on actual device i.e. on iPhone.
So I wonder that why it is necessary to to test my iPhone app on actual iPhone device though they (Apple) have given "simulator" which is near about same as my device?
You need to test application on real device to at least to look how it behaves with:
iOS developers, please continue this list.
One thing you'll never know when testing with emulator is how it really feels for a user holding a real device in their hand, sliding fingers on its screen. As a result, user actions that seemed smooth when simulating with touchpad on your laptop, may turn out quite cumbersome for a real device usage. To make sure that your application is OK, test it with real device.
Another thing worth testing with real device is battery consumption. That one is really safer to just test with real device than rely on how well simulator developers managed to reproduce it in their tool.
There may be other things that just aren't close enough at simulator. Audio volume and balance for example - the way it sounds on your laptop, may differ from how it will be on a real phone. Vibration is another example that is barely possible to get right with simulator. The way how gyro sensors work at the real phone. GPS / location related stuff. Etc etc etc...
Simulator vs real device testing is quite an important issue. In one of my past projects, an important part of commercial success was a careful balancing between these kinds of testing, which essentially boils down to permanently re-asking questions like why on device?
Real work starts when one asks why, picking the reasons to choose between on-device and simulator testing in particular cases and situations.
Ignoring device testing exposes your product to a (pretty high) risk of breaking right in the hands of end user, completely destroying all the efforts you put into development. But thing is, simulator testing is much MUCH cheaper and much easier for automation. If one blindly sticks to on-device-only testing, their releases might become substantially later and more expensive than those of competitors.
From experience, and a take off from the best voted answer:
Your fingers instead of the mouse was the largest difference when we developed Decimation X2 for the Windows Phone 7. It was coded on an emulator, as we did not have a WP7 and it was before the WP7 was released. We could have potentially received a free WP7 before release, in case that last sentence made no sense to you, as we were invited by Microsoft to have a launch title on their phone. It turned out that, what we wanted the user to do with their fingers was very hard on the real phone, but easy with a mouse. And it had to do with screen-edge cases. Unfortunately our game required your fingers to be at the screen-edge all the time, which some phones made hard to do due to their sunken screen as well as thick cases. We actually put out a patch to fix this. This means all our first-time users were shown a bad, and potentially unusable, version of our game. :(
Different hardware speed was the next largest difference. We had to literally guess by the inaccurate method of taking our Xbox 360 version of the game and downgrading it accordingly to half the frame-rate (60fps to 30fps) and one-third the GHz (3.0GHz to 1.0GHz), and we guessed wrong. The processors were different, of course, and we knew this. Without the hardware, we were left with lame guesswork. It was not our choice as we had no WP7, but we learned the lesson I'm sharing with you now. On some phones, during the most intensive parts of the game, it dropped frames. :( Nobody seemed to care as they assumed the slowdown was appropriate for such intense parts. So it was no big deal, but the point is: Had this been a big deal, the "app" would have been broken from our lame guesswork.
Test on real hardware. And when you code for various phone hardware, test on the lower end ones if performance is an issue.
The iPhone Simulator implements some APIs that the iPhone itself does not (The main one that comes to mind being the DOM XML API, where the iPhone only supports SAX to my knowledge, this might have changed though now.)
It will also let you 'feel' the app, are your buttons the right size? Do the right buttons fall under one's thumb? Is the iPhone up to running the app? The simulator is not an emulator and works by just adding Cocoa Touch to your desktop Mac for that app. You have to simulate out of memory warnings and the like.
Because you are not going to have a lot of users walking around with a simulator in their pocket.
EDIT: Whenever you're testing your app on a simulator (or emulator), you're using a fake device that cannot be, by definition, a 100% accurate representation of the real thing. An emulator might be more accurate than a simulator, but there will still be differences. The only 100% accurate emulator is the device itself.
Designing, testing, optimizing code on a simulator results in an application that is finely tweaked to optimally work on a.. simulator. Your users will not have simulators, though; you're targeting the wrong device. A very similar one; but not the very same device that your users will use.
This can lead to several kind of issues. Severe issues like bugs, crashes are certainly your top priority. But there are others; such as ergonomics. Try to hold the Simulator in your hand. Just try. UI elements are rendered on a different screen, with possibly different color rendering, and certainly different dimensions (a problem exacerbated by the beautiful retina displays, not entirely solved by also using a retina macbook, for instance). Are those exquisite shades of gray equally distinguishable on a device in the sun?
Subtle differences in speed, and different sensor emulation (or lack thereof), can sometimes dramatically alter the experience.
If your application relies on internet connectivity, there is no way for you to switch between LTE, 3G, EDGE or GPRS, to test different scenarios, or even test different carriers.
Are you going to support jailbroken devices? Perhaps you aren't, but if you are, you're probably willing to test your app with one. Or, if you aren't, are you confident that you're detecting a jailbroken environment?
Is the iPad game you're developing on the simulator equally usable when a user is holding its weight and using her fingers to play? Are unintended multiple touches able to break your app, something that you weren't able to anticipate in the safe single-touch (or symmetric double-touch) simulator environment?
Would you be comfortable boarding a plane commanded by a pilot who has never actually left the ground?
The bottom line is: before shipping, please do use the same device that your users are going to use. None of them is going to use a Simulator.
practical reason:
1) You do not have "send mail" functioning.
2) You cannot put device upside-down..
and of course already said reason:
3) low bandwidth
4) very small computational power compared to simulator
5) Open GL calls are implemented a bit different in simulator
6) disk space / RAM..
While it's been mentioned that hardware's performance is generally worse, it must be noted that this is not the case with OpenGL ES. The simulator implements it in software so it is not uncommon, to notice huge performance increase while running on the device itself.
Moreover, there are some minor differences between the software and hardware implementations of Open GL ES, e.g. shader precision hints might have different outputs.
When we implement things for iOS (or android, or windows phone), we develop not for the desktop but for the device. For some calculation/resource-heavy applications, this may mean to normal behavior on the simulator BUT issues on the actual device.
So situations like this can be encountered in later stages if we do not test on device from the beginning:-
There are some features like PushNotification, using the camera, etc. which we can only test on a device; these are features that can't be tested on a simulator.
And testing your app on an actual device before submission to the App Store reduces the chances of app rejection; sometimes an app works fine in the simulator but it crashes on an actual device, which is the main reason for app rejections.
There can be real performance differences between the actual device and the emulator. We found that only testing with the emulator resulted in a very slow app in many cases, which we didn't expect.
The user experience varies from one device to another due to different OS and
hardware specifications. Hence, it is necessary to test an iPhone app on the actual
devices – mobile devices running various versions of iOS in the marketplace.
Although a simulator is useful in identifying any issues faced by the end-user,
testing the app on the original device will help in identifying and addressing major
concerns of the user.
