fzyzcjy / flutter_convenient_test

flutter_convenient_test Write and debug tests easily, with full action history, time travel, screenshots, rapid re-execution, video records, retryability, interactivity, isolation and more. 🚀 Advantages • **Enjoy 5x speedup**: Able to execute your tests not only on simulator/device, but also on host computer (much faster), without changing any code. • **Full action history**: Gain insight on what has happened to the tests - View all actions/assertions taken in the tests, with friendly descriptions. • **Time travel, with screenshots**: What did the UI look like when that button was tapped 50 steps ago? Now you know everything • **Rapid re-execution**: Edit code, save, run - done within seconds, not minutes. • **Videos recorded**: Watch what has happened, in full detail. • **Retryability**: No manual , wait or retry anymore. Just write down what you want, and the framework will retry and wait. • **Being interactive**: Play with the app interactively, again within seconds. • **Isolated**: One test will no longer destroy environment of other tests - run each with full isolation (optional feature). • **Abundant information for errors**: When something is not working, it tries to print out as much information as possible to help you quickly debug. • **Enhanced golden**: GUI to check golden changes, allow a few pixels to be different, screenshot widgets above your widget, etc (Standalone feature, does not require full convenient_test) • **Utilities**: Enter text without replacing, pump with run async, etc. (Standalone feature, does not require full convenient_test) And also... • ** still there**: Freely use everything from , , , etc. • **Flaky test awareness**: No false alarms, while no blindly ignoring. • **Suitable for CI**: Run headlessly, with logs to be visualized when needed. • **Miscellaneous**: Visually see target regions, run single test/group within seconds, raw logs... 📷 Quick demo **NOTE**: The video only contains features up to 2022.05. New features not included in the video: The 5x speedup, the enhanced golden, etc. https://user-images.githubusercontent.com/5236035/167066810-d0aa36ba-0113-4140-92f9-cec0a9e77ed1.mov Stability It has been used extensively in my own 200kloc Flutter app in production environment, and I integrate it with my CI infra so it is run everyday. P.S. Surely, this package - especially the doc - is not yet perfect. If having problems, just create an issue and I usually reply quite quickly. Unreleased features I have been using these components internally for a long time, but have not found time to make cleanup (e.g. remove internal dependencies) in order to open-source it. If you like it, feel free to tell me and I will prioritize it! • Monkey tester: Randomly interact with tappable/draggable/... regions of your app, trying to find crashes. I even find a bug of Flutter *itself* using this simple monkey! • Benchmark helper: Using one command to automatically benchmark on all devices, upload to storage, and generate reports. (I run it in pseudo-CI and read the reports to see performance regressions.) 📚 Features 5x speedup & remove mobile devices In addition to the standard way of integration tests, i.e. run them on a real mobile device / a simulator, this package supports running the exact same testing code on your host computer (Windows/MacOS/Linux) with similar experience. This leads to a big speedup and reduction of needed resources. As we all know, running code on simulators / real devices are slow, and hard to parallelize as well. Running on host computer, on the other hand, does not have this problem. Remark: In my own app in production, it is >10x faster, but without creating public and reproducible tests, I do not want to claim such a big improvement - so test by youself and see how fast it provides :) How to use it: Just write code normally, and execute your code *as if* it is a widget test. If you see weird fonts, you may want loadAppFonts. Full action history > See actions/taps/assertions taken in the tests, with friendly descriptions (in the left panel) Time travel with screenshots > Tap an action to see its screenshots P.S. Use mouse wheels and dragging to zoom and pan the screenshots. P.S. The gif is outdated - new version has (1) an overview of all screenshots (2) a ruler for each screenshot. Rapid re-execution > Edit code, save, run - within seconds Videos recorded > Watch what has happened in full detail (in right panel) P.S. Highlighted action is in sync with the playing video. Tap action to seek video. Being interactive > Temporarily play with the app, interactively. (Flutter originally cannot interact with app in test mode) Retryability No manual , wait or retry anymore. Just write down what you want, and the framework will retry and wait. Example: More in quickstart and tutorials below. Isolation mode One test will no longer destroy environment of other tests - now you can run run each with full isolation. This is especially helpful in big projects (such as mine), when an end-to-end setup is not easy to tear down, and one failed test can cause all subsequent tests to have damaged execution environment and thus fail. Technical details: If this mode is enabled, a hot restart will be performed after each attempt of each test. Abundant information for errors When something is not working, it tries to print out as much information as possible to help you quickly debug. For example, suppose you provide a that accidentally matches two widgets. Then, convenient_test will tell you the location of each widget, as well as a chain of render objects. Together with the screenshots/videos it provides, the finder problem can be fixed easily. Remark: The GIFs are outdated and does not show this feature. Enhanced golden This is a standalone feature, i.e. you can use it without using convenient_test. Golden overview GUI It is a simple GUI to list and check all golden differences, with interactivity like pan and…