Picture-driven programming

I would really like to know how this works. In essence, treating the entire visual interface of a program as data input to another program that modifies the original behavior, and another indication that visual programming is happening.

This concept is immensely powerful, though not unprecedented (numerous GUI testing tools exist which purport to record a users visual actions and play them back). What is striking is the generalization of the concept – it’s not aimed specifically at software testing, but at any task that benefits from remixing or automating the original software.

To me, the weak link in this research is the concept of scripts. We already have far too many scripts, scripting languages, scripty whatever you wills... This metaphor is dragging the tech industry kicking and screaming towards a mass apocalypse of scrapheap programming. What we should really be asking for is the end of coding as we know it, removing the concept of applications, and instead, building software by connecting and combining simple transformations.