SOFTWARE TESTING

irl

APP STORE SPECIFIC CHECKS

#	Description
4.1	Application cannot use "non-public APIs". This means that you are not you can use some functions, which the platform uses for its own applications. This can usually be done check automatically, for example, when help http://www.chimpstudios.com/appscanner/)
4.2	Application cannot reprogram control keys of the device, not intended for such use (for example, use volume button for taking photos).
4.3	The application should not have access to files outside the application without user permissions (e.g. copy the address book or receive information from other applications).
4.4	The application should not have access to folders and files outside the container directories and documents.
4.5	Application cannot load code for installations without user consent.
4.6	The app can only be updated through application store.
4.7	After downloading, the application should be workers. It shouldn't shut down after several days.
4.8	The application may not be "experimental", "beta", "demo" or "test" version.
4.9	Apple product names must be listed without typos (iPhonez - incorrect).
4.10	If the application is using the network, it will not uses it through third parties (non-Apple) browsers.
4.11	The appendix must not mention third party platforms (for example, "Also available for Android! ").
4.12	Application cannot use outdated interfaces (e.g. wheel iPod).
4.13	Application multitasking functionality can only be accessed by its original purposes (i.e. for VoIP, audio playback, positioning, completion tasks, local notifications, etc.). it means that in general the application does not can run in the background and should be closed if not in use.
4.14	The application should contain at least some functionality. It cannot consist of one page and text, cannot be just a song, movie or book - for this there are other platforms.
4.15	Functionality must comply description in the app store.
4.16	In general, the application should be "worthy". Explicit material - sex, violence, drugs, alcohol, tobacco - not must be displayed in it, and it is not should speak about individual ethnic or religious groups derogatory.
4.17	The application must be honest. His the description must be true, and the whole the functionality should work like described. If the application gives diagnostic information, it should be reliable. This also applies to the genre and categories in the store. Application icons must match and suit him.
4.18	Application cannot restrict users in the choice of geolocation or mobile operator.
4.19	The application cannot send spam, spread viruses, or use other Apple platforms (e.g. Game Center / Push Notifications) for these purposes.
4.20	An application should strive to store the minimum amount of data in iCloud. All, stored in iCloud must be created user. Information you can download or restore, should not get into iCloud.
4.21	Application cannot use geolocation services without permission.
4.22	All references in the application code must work.
4.23	Application cannot use user location without permissions.
4.24	Geolocation services cannot used for autonomous control vehicles or ambulance calls help.
4.25	Application cannot use notifications without the user's consent.
4.26	Notices must be sent through Apple Push Notification API using APN ID.
4.27	Notifications should not contain personal data.
4.28	The application cannot distribute personal user information (e.g. ID player) via Game Center.
4.29	Banner ads should be hidden if advertising is not available.
4.30	The application must respect copyright Apple and others.
4.31	In-App Purchase Engine Cannot be used to purchase goods and services used outside the application.
4.32	The in-app shopping engine cannot be used to raise funds for charity (there is an SMS for this).
4.33	The in-app shopping engine cannot used to buy lottery tickets directly from the application.
4.34	Apps that reward the user use the device so that it can be damaged will not be approved.
4.35	The application cannot request personal user data (e.g. email) to function.

irl

INTERFACE CHECKS
This checklist is based on guidance from Apple and other experts. It does not cancel testing
usability - it is much more reliable for understanding user experience.

#	Description
3.1	Control elements maximum unobtrusive (for example, they disappear, if notare used for a long time).
3.2	User can return to the previous screen (for example, by pressing "Back" or "Cancel").
3.3	The main function of the application is easy to understand and self-evident.
3.4	The function most likely to be use user, highlighted (to for example, on iOS, a blue button means default / most likely action act).
3.5	Minimize user actions, using dropdown lists or tabular view where the user can make a choice instead of manually entering data.
3.6	The user should not be able to store data locally, outside the sandbox applications.
3.7	The user should not be able to view access levels for files.
3.8	The search function should be available for long lists to scroll through.
3.9	Add a progress icon ("Loading ...") for situations with low performance, preferably with a clear message.
3.10	In case of "live" data filtering when entering a search term check application performance.
3.11	Appearance of buttons for standard actions familiar to the user (for example, update, sort, cart, reply, back etc.).
3.12	Standard icons are not used for functions for which they are not normally apply.
3.13	The application correctly responds to the shift device orientation.
3.14	Active screen elements should be approximately 7x7 mm in size. Use the pixel density of the target device to calculate the number of required pixels (see. section "Documentation")
3.15	Do not redesignate gestures that perform standard functions (for example, swiping from top to bottom reveals the center notifications).
3.16	The login prompt does not appear in application until it becomes necessary.
3.17	If the application is stopped unexpectedly, user data should be stored locally and available when start the application.
3.18	When deleting documents, the user warned of the consequences.
3.19	Keyboard adjusts to expected input (e.g. numbers / letters).
3.20	Is it easy to distinguish between inactive buttons and active?

irl

APP FEATURES

#	Description
2.1	Has it been tested on different devices / OS versions?
2.2	Stability check: if the app has lists (e.g. images), try flipping through them quickly.
2.3	Stability check: if the app has lists (like images), try flipping through to positions "before the first image" and "after the last".
2.4	Does the application stop downloading if it exceeds size allowed in OS for download via mobile the Internet?
2.5	Integration: Is It Connecting To Social Media Correctly (LinkedIn, Twitter, Facebook, etc.).
2.6	The application does not interfere with the operation of other applications in background mode (GPS, music playback, etc.)
2.7	Can I print from application (if applicable)
2.8	Search functionality displays relevant results.
2.9	Common control gestures work application.
2.10	What happens when you select multiple options at the same time (unplanned multitouch - for example, choosing multiple contacts from the notebook at once).
2.11	Application name must be "speaking"
2.12	Does the application limit the cache, cleans it?
2.13	Reloading data from a remote service configured not to cause problems with server side performance (manual reboot can reduce the number of requests to server).
2.14	Does the application go to sleep when it is running background (this saves battery power)?

irl

NETWORK CHARACTERISTICS

#	Description
1.1	Is application behavior consistent desired if it is connected to the Internetvia Wi-Fi?
1.2	Whether application behavior is consistent desired if it is connected to the Internet via 3G?
1.3	Does application behavior match desired if it is connected to the Internet through 2G?
1.4	Is Application Behavior Compliant desired if the network is not available?
1.5	Does the application resume when accesses the network again after interrupted access?
1.6	Updating transactions is correct after reconnecting to the network.
1.7	Does the application continue as expected work if it is tied or in any way otherwise connected to another device?
1.8	What happens if the application switching between networks (Wi-Fi, 3G, 2G)?
1.9	Does the application use standard network ports (Mail: 25, 143, 465, 993 or 995, HTTP: 80 or 443, SFTP: 22) for remote connections: some providers block individual ports.

irl

Black box testing will never go away as long as human beings are using your product. There is no substitute for the human ability to identify "something weird going on".

irl

TDD is Test Driven Development. This means writing a test that fails because the specified functionality doesn't exist, then writing the simplest code that can make the test pass, then refactoring to remove duplication, etc. You repeat this Red-Green-Refactor loop over and over until you have a complete feature.

BDD is Behavior Driven Development. This means creating an executable specification that fails because the feature doesn't exist, then writing the simplest code that can make the spec pass. You repeat this until a release candidate is ready to ship.

In BDD there is a special persons (testers, product owners, others) who writes the scenarios in feature text files:
Here is a sample using BDD and Gherkin syntax:

Feature: Account Holder withdraws cash

  Scenario: Account has sufficient funds
   Given the account balance is $100
     And the card is valid
     And the machine contains enough money  
    When the Account Holder requests $20
    Then the ATM should dispense $20
     And the account balance should be $80
     And the card should be returned

Programmers have special frameworks (for example, cucumber, specflow) that translate such scenarios into tests written using programming language, develop the tests.

What is the advantage of BDD from TDD?

Tests readable for non-programmers.
They are easy to change. They are often written in almost pure English.
They can now be written by the product owner or other interested parties.
The test results are more "human".
Tests are independent of the target programming language. Migration to another language is greatly simplified.

The Difference are:

BDD uses specific almost pure English words syntax, but TDD is written with code used in development
In BDD, a test is written that can satisfy both the developer and customer, but in TDD you write a test that will only satisfy a developer and the code they write

BDD has actually evolved from TDD, as a way to eliminate the shortfalls of TDD. So there is absolutely no harm in implementing both approaches – one to support the quality of the code the developer writes, and the other to support the behavior of the system defined by the product owner.

irl

Ussually I use one of the tools:

irl

APP STORE SPECIFIC CHECKS

#	Description
4.1	Application cannot use "non-public APIs". This means that you are not you can use some functions, which the platform uses for its own applications. This can usually be done check automatically, for example, when help http://www.chimpstudios.com/appscanner/)
4.2	Application cannot reprogram control keys of the device, not intended for such use (for example, use volume button for taking photos).
4.3	The application should not have access to files outside the application without user permissions (e.g. copy the address book or receive information from other applications).
4.4	The application should not have access to folders and files outside the container directories and documents.
4.5	Application cannot load code for installations without user consent.
4.6	The app can only be updated through application store.
4.7	After downloading, the application should be workers. It shouldn't shut down after several days.
4.8	The application may not be "experimental", "beta", "demo" or "test" version.
4.9	Apple product names must be listed without typos (iPhonez - incorrect).
4.10	If the application is using the network, it will not uses it through third parties (non-Apple) browsers.
4.11	The appendix must not mention third party platforms (for example, "Also available for Android! ").
4.12	Application cannot use outdated interfaces (e.g. wheel iPod).
4.13	Application multitasking functionality can only be accessed by its original purposes (i.e. for VoIP, audio playback, positioning, completion tasks, local notifications, etc.). it means that in general the application does not can run in the background and should be closed if not in use.
4.14	The application should contain at least some functionality. It cannot consist of one page and text, cannot be just a song, movie or book - for this there are other platforms.
4.15	Functionality must comply description in the app store.
4.16	In general, the application should be "worthy". Explicit material - sex, violence, drugs, alcohol, tobacco - not must be displayed in it, and it is not should speak about individual ethnic or religious groups derogatory.
4.17	The application must be honest. His the description must be true, and the whole the functionality should work like described. If the application gives diagnostic information, it should be reliable. This also applies to the genre and categories in the store. Application icons must match and suit him.
4.18	Application cannot restrict users in the choice of geolocation or mobile operator.
4.19	The application cannot send spam, spread viruses, or use other Apple platforms (e.g. Game Center / Push Notifications) for these purposes.
4.20	An application should strive to store the minimum amount of data in iCloud. All, stored in iCloud must be created user. Information you can download or restore, should not get into iCloud.
4.21	Application cannot use geolocation services without permission.
4.22	All references in the application code must work.
4.23	Application cannot use user location without permissions.
4.24	Geolocation services cannot used for autonomous control vehicles or ambulance calls help.
4.25	Application cannot use notifications without the user's consent.
4.26	Notices must be sent through Apple Push Notification API using APN ID.
4.27	Notifications should not contain personal data.
4.28	The application cannot distribute personal user information (e.g. ID player) via Game Center.
4.29	Banner ads should be hidden if advertising is not available.
4.30	The application must respect copyright Apple and others.
4.31	In-App Purchase Engine Cannot be used to purchase goods and services used outside the application.
4.32	The in-app shopping engine cannot be used to raise funds for charity (there is an SMS for this).
4.33	The in-app shopping engine cannot used to buy lottery tickets directly from the application.
4.34	Apps that reward the user use the device so that it can be damaged will not be approved.
4.35	The application cannot request personal user data (e.g. email) to function.

irl

INTERFACE CHECKS
This checklist is based on guidance from Apple and other experts. It does not cancel testing
usability - it is much more reliable for understanding user experience.

#	Description
3.1	Control elements maximum unobtrusive (for example, they disappear, if notare used for a long time).
3.2	User can return to the previous screen (for example, by pressing "Back" or "Cancel").
3.3	The main function of the application is easy to understand and self-evident.
3.4	The function most likely to be use user, highlighted (to for example, on iOS, a blue button means default / most likely action act).
3.5	Minimize user actions, using dropdown lists or tabular view where the user can make a choice instead of manually entering data.
3.6	The user should not be able to store data locally, outside the sandbox applications.
3.7	The user should not be able to view access levels for files.
3.8	The search function should be available for long lists to scroll through.
3.9	Add a progress icon ("Loading ...") for situations with low performance, preferably with a clear message.
3.10	In case of "live" data filtering when entering a search term check application performance.
3.11	Appearance of buttons for standard actions familiar to the user (for example, update, sort, cart, reply, back etc.).
3.12	Standard icons are not used for functions for which they are not normally apply.
3.13	The application correctly responds to the shift device orientation.
3.14	Active screen elements should be approximately 7x7 mm in size. Use the pixel density of the target device to calculate the number of required pixels (see. section "Documentation")
3.15	Do not redesignate gestures that perform standard functions (for example, swiping from top to bottom reveals the center notifications).
3.16	The login prompt does not appear in application until it becomes necessary.
3.17	If the application is stopped unexpectedly, user data should be stored locally and available when start the application.
3.18	When deleting documents, the user warned of the consequences.
3.19	Keyboard adjusts to expected input (e.g. numbers / letters).
3.20	Is it easy to distinguish between inactive buttons and active?

irl

APP FEATURES

#	Description
2.1	Has it been tested on different devices / OS versions?
2.2	Stability check: if the app has lists (e.g. images), try flipping through them quickly.
2.3	Stability check: if the app has lists (like images), try flipping through to positions "before the first image" and "after the last".
2.4	Does the application stop downloading if it exceeds size allowed in OS for download via mobile the Internet?
2.5	Integration: Is It Connecting To Social Media Correctly (LinkedIn, Twitter, Facebook, etc.).
2.6	The application does not interfere with the operation of other applications in background mode (GPS, music playback, etc.)
2.7	Can I print from application (if applicable)
2.8	Search functionality displays relevant results.
2.9	Common control gestures work application.
2.10	What happens when you select multiple options at the same time (unplanned multitouch - for example, choosing multiple contacts from the notebook at once).
2.11	Application name must be "speaking"
2.12	Does the application limit the cache, cleans it?
2.13	Reloading data from a remote service configured not to cause problems with server side performance (manual reboot can reduce the number of requests to server).
2.14	Does the application go to sleep when it is running background (this saves battery power)?

irl

NETWORK CHARACTERISTICS

#	Description
1.1	Is application behavior consistent desired if it is connected to the Internetvia Wi-Fi?
1.2	Whether application behavior is consistent desired if it is connected to the Internet via 3G?
1.3	Does application behavior match desired if it is connected to the Internet through 2G?
1.4	Is Application Behavior Compliant desired if the network is not available?
1.5	Does the application resume when accesses the network again after interrupted access?
1.6	Updating transactions is correct after reconnecting to the network.
1.7	Does the application continue as expected work if it is tied or in any way otherwise connected to another device?
1.8	What happens if the application switching between networks (Wi-Fi, 3G, 2G)?
1.9	Does the application use standard network ports (Mail: 25, 143, 465, 993 or 995, HTTP: 80 or 443, SFTP: 22) for remote connections: some providers block individual ports.

irl

The issue was resolved by using UI Automator

irl

I need to write a test that would create a file on a usb device and check for its existence (for example). But in order to interact with a usb device, you need to obtain interaction rights.

How can I get permission to interact with usb in my unit tests?

irl

I do it every day when look at commits of developers in github

irl

@irl

pages:account/best, irl

pages:account/latest-posts, irl