Automation & Testing

Sikuli Bot
SikuliBot is an image-based approach to automating user interface. SikuliBot extends the visual programming concept of Sikuli Script from the graphical UIs to the real world of physical UIs, such as mobile devices' touch-screens and hardware buttons. The key to our approach is using a physical robot to see an interface, identify a target, and perform an action on the target using the robot's actuators. We demonstrate working examples on MakerBot 3D printer that could move a stylus to perform multi-touch gestures on touchscreen to automate tasks such as swipe-to unlock, playing a virtual piano, and playing the Angry Bird game. A wide range of automation possibilities are made viable using a simple scripting language based on images of UI components. The benefits of our approach are: generalizability, instrumentation-free, and high-levelprogramming abstraction. ( UIST '14)

G-code Simulator
G-code is the machine language that controls CNC millers, 3D printers, laser cutters, etc., moving motors in X,Y,Z-axis. Once people download STL files online, it is loaded into 3D printer and translated into the G-code, by tons of lines of commands for extruding, changing speeds of header movements, moving header to designated position. From this class project, I programmed the graphics G-code parser that reads raw G-code, and simulate 3D printer's performance in screen. This can be used to verify 3D models, extensively checking how 3D printer works.