Difference between Fitnesse and Robot framework
briley last edited by
We are choosing what system to start using in our company. it should be used for both backend (REST API, some DB checks) and UI testing it should use a simple language so even non-programmers/tester can understand the test cases (Product Owners should be able to see whether all acceptance criteria are covered) it should support integration with Jenkins it should support versioning of test cases so that for a particular product version we also can check out relevant test cases right now we use TestRail (test case management SW); so would be nice if it integrates with it (at least it is possible to program it so send test results there) or completely replaces it Any ideas, experience? Thanks
Overview Robot framework is an excellent choice that meets all of your goals. Robot can be used for UI tests (via selenium), REST and SOAP service tests, database tests, and just about any other type of acceptance test. You can even use robot tests to improve your manual testing process. Robot is keyword driven Robot is keyword-driven, which makes it very easy to create test cases that can be understood by product managers and stakeholders, without them needing to learn a programming language. Also because it is keyword driven, it makes it easier for QA professionals with limited technical expertise to write tests. What I like about robot over some similar tools such as cucumber is that you have the choice to write BDD-style tests (given/when/then), procedural tests, and data-driven tests. You aren't locked into a single testing strategy -- you can choose the style that best fits each scenario. Robot is highly extensible Robot is highly extensible, in python, java, and/or any .NET language. In fact, you can use just about any language at all through the remote API that robot has. The developers on your team can participate in testing by writing the more difficult keywords in a programming language while letting the QA people focus on test scenarios, coverage, etc. Robot integrates with development tools Because tests are plain text, they integrate extremely well with version control systems. They diff and merge nicely, especially if you use the plain-text, pipe-delimited format. There is a robot-specific jenkins plugin, and has a command line test runner, so it is extremely easy to integrate with Jenkins. Also because the tests are plain text, your test writes (both developers and QA) can use whatever tool they are most comfortable with. Developers can use emacs, visual studio, eclipse, etc. non-developers can use brackets, notepad++, sublime text, etc. Many of these editors have robot-specific programming modes that provide syntax highlighting and other features. You aren't forced to use a specialized tool. Robot has robust reporting Robot generates a very simple-to-parse xml output file, and has an option to generate an xunit-style output file. This makes it pretty easy to integrate with other tools. It also has an interface that can call python functions for various test events -- for example, every time a suite finishes. Since testrail has a web-based API, it would be easy to set it up so that every time a suite finishes, it can send the results to testrail. Other advantages Some people prefer a real programming language when interacting with selenium, and that's certainly a great way to go if your team has the skill to do that. Most QA teams don't have that luxury. Even if you do, there are other advantages to using a framework like robot: built-in reporting, a fabulous tagging mechanism, and other features that just don't come with pure programming environments such as the robot listener interface. Summary At the time that I write this I've used robot framework at three different organizations. For one of those it wasn't an organizational fit and was eventually abandoned. Developers did a lot of the testing and they wanted a real programming language. In the other two, there were more dedicated QA who write tests, and for both of those organizations, robot worked extremely well and was embraced by the entire organization -- POs, developers and QA alike. Is it the best tool? No, there is no best. It is, however, extremely flexible, extremely extensible, and is a very effective testing tool.