@zymir l'm not sure if you're adopting Jira cloud or Jira datacenter, as the options you have will depend on that.
Anyway, let me give you some food for thought
I work at Xray, but I'll do my best to provide some useful information (non-sales related).
I will answer inline, below.
integration with JIRA
Both Xray and Zephyr (squad and scale) work inside Jira, so therefore their integration is native => the most complete you can have.
However, some teams may need just to have a way to upload test results to Jira and see their impacts on the related stories. If that's the case, then TestRail is a viable option.
Here, you'll have to think if you want people (devs/testers) to use one tool, or several tools.
ability to import existing test cases from Excel sheets
In the case of Xray, it provides a built-in Test Case Importer that you can use to import test cases in bulk.
I won't answer this one on purpose; that's something you have to evaluate.
ability to integrate automated tests
Xray supports a bunch of test automation tools/frameworks/libraries, including Junit, TestNG, xUnit, Nunit, Cucumber, Cypress, Playwright, Postman, etc. You may find around dozens of test automation tutorials and integrations with CI/CD tools in Xray docs.
quality of the JIRA plugin
I would recommend you to look at:
- reviews on Atlassian Marketplace, for example for Xray cloud
- frequency of releases, and how frequent are bugfixes and new releases. You can see it also on Atlassian Marketplace, on the Versions tab>pick datacenter>then see all versions under Resources section, which shows info both for the product on Jira cloud and on Jira server/datacenter; example for Xray
in case of Zephyr and Testrail, is it advisable to use the JIRA plugin only without using their enterprise service?
Please check what each vendor provides exactly with their enterprise offering and see if it matches your needs; also check the SLAs.
Some organizations demand priority support, customer representative, 24x7 support, etc.. check exactly your needs and what is offered by the the enterprise offering. I would also consider the frequency of releases and the policy of bugfixes as something to consider.. because you can have a "enterprise kinda of service" but bugfixes could take a while to be released.
It would be good if someone could provide a Matrix Comparison for these tools based on these features.
Xray has this comparison page, which is not complete.
I'll give you some few things that you can use to help you pick tools in this space. Please don't consider this an exhaustive list
- what testing approaches (manual test cases, test automation, exploratory testing) are we following? does it support all of those?
- are our teams working in siloed environments? Or are they part of the scrum / agile team, or even embed as skills within the team? Siloed teams may prefer to have separate tools (even though I don't think it promotes collaboration, transparency, agility)
- do we need to track coverage? and full, bi-directional traceability between epics<=>stories<=>tests<=>runs<=>defects?
- do we need to track the status of our product on multiple versions/devices/environments at the same time?
- is the product flexible to enought so we can adapt processes and tooling, including test automation frameworks and CI/CD tools, at any time?
Hope this helps, anything please let me know