Python Mouse Click and Move Mouse on Apple Mac OSX Snow Leopard 10.6.x

For anyone interested I recently had to write a touchscreen wrapper for Apple Mac OSX 10.6 Snow Leopard, and could not find any clean simple examples out there on the net and it was quite frustrating.

what i had to do was;

- takes serial data in MS-HID format from the screen

- decodes that into x,y screen resolution co-ordinates for mouse move, mouse up, mouse down events

- generates those mouse events in OSX automatically

Working as a driver and written in python on osx snow leopard.

Now, Here is how;

:: a simple mouse move and click (mouse down then mouse up) ::

##############################################################
#
#               Python OSX MouseClick
#
#       (c) 2010 Alex Assouline, GeekOrgy.com
#
##############################################################
import sys
try:

        xclick=int(sys.argv[1])
        yclick=int(sys.argv[2])
        try:
                delay=int(sys.argv[3])
        except:
                delay=0
except:
        print "USAGE mouseclick [int x] [int y] [optional delay in seconds]"
        exit()

print "mouse click at ", xclick, ",", yclick," in ", delay, "seconds"

# you only want to import the following after passing the parameters check above, because ‘importing’ takes time, about 1.5s
# (why so long!, these libs must be huge : anyone have a fix for this ?? please let me know.)

import time
from Quartz.CoreGraphics import CGEventCreateMouseEvent
from Quartz.CoreGraphics import CGEventPost
from Quartz.CoreGraphics import kCGEventMouseMoved
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseDown
from Quartz.CoreGraphics import kCGEventLeftMouseUp
from Quartz.CoreGraphics import kCGMouseButtonLeft
from Quartz.CoreGraphics import kCGHIDEventTap

def mouseEvent(type, posx, posy):
        theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
        CGEventPost(kCGHIDEventTap, theEvent)

def mousemove(posx,posy):
        mouseEvent(kCGEventMouseMoved, posx,posy);

def mouseclick(posx,posy):
        #mouseEvent(kCGEventMouseMoved, posx,posy); #uncomment this line if you want to force the mouse to MOVE to the click location first (i found it was not necesary).
        mouseEvent(kCGEventLeftMouseDown, posx,posy);
        mouseEvent(kCGEventLeftMouseUp, posx,posy);

time.sleep(delay);
mouseclick(xclick, yclick);

print "done."

Then we want to take the serial data from the display which is some chinese 19 inch touch screen like galaxy something – i use an FTDI 232 cable and ftdi VCP osx drivers to pull the data (works very well with python and pySerial) — uses virtually nothing of the cpu on the mac mini which runs it.

If you want the actual touchscreen/data code just ask me, its too long to paste here and may certainly need tweaking to your display model/data.

Note1: I am not sure what prerequisits must be in place for the above to work, i have run it on out-of-the-box mac mini’s without anything extra installed at all, and its fine.

Note2: There was a deprecated method in OSX Python called ‘CGPostMouseEvent’ : DONT USE IT! – it led to a lot of strange behavior and crashes on my systems. The above ‘CGEventCreateMouseEvent’ in the code is the latest proper method to use.

Comments

  • Tony says:
    8-06-10

    Nice! I needed to move the mouse to prevent sleep, and while there is a free small app available (Jiggler), your method is the best I’ve seen in 2 hrs of googling.

  • Alex says:
    8-06-10

    Cheers, im glad it worked for you Tony.

    I too searched for many hours looking for a simplified straight to the point working script.

    you can simply use mousemove instead of mouseclick to prevent sleep, might be better for you.

    also if you change the last bit of code:

    time.sleep(delay);
    mouseclick(xclick, yclick);

    to this:

    while 1:
    time.sleep(delay);
    printing “clicking mouse at”, xclick, yclick
    mouseclick(xclick, yclick);

    then it will loop forever clicking the mouse every ‘delay’ seconds, preventing sleep permanently.

  • Tony says:
    8-06-10

    Yes, that’s what I eventually did. I just used the mousemove from your code.

    Is there a way to save the current mouse position first? (so that I can return the mouse pointer back to where is started). Not really necessary, but just wanted to know.

    BTW, when I first copied your code, I got an error “SyntaxError: Non-ASCII character ‘\xe2′” from the quotes from” ‘importing’ ” in your comment. Might be an issue with my editor (Smultron).

    (and I was able to just put the code in my bash script with a Here Doc. Nice and simple now — Thanks again!)

  • Tony says:
    8-06-10

    Got it. To save and reset the mouse position, add:

    from Quartz.CoreGraphics import CGEventCreate
    from Quartz.CoreGraphics import CGEventGetLocation

    Then to save location:
    ourEvent = CGEventCreate(None);
    currentpos=CGEventGetLocation(ourEvent);

    and to restore:
    mousemove(currentpos.x,currentpos.y)

  • Alex says:
    8-06-10

    Thanks Tony, thats great, nice one!.

  • Matt says:
    8-06-10

    I have virtually no programming/coding experience, so please bear with me while I ask some possibly stupid questions:

    I’m planning on getting a 7″ USB touchscreen to use with OS X, do you know how simple it would be to use the above code to create a driver? (Like I said, daft questions, I’m just a bit stuck).

    The screen I was planning on getting is supplied with drivers for Snow Leopard, but I found this post because of the comments from Tony about saving the current position of the mouse pointer.

    I’m creating a touchscreen application launcher, so don’t want to have to move the mouse back from the screen each time it is used.

    If you have any advice, it would be greatly appreciated.

  • Alex says:
    8-06-10

    Hi Matt,

    Sure that should not be very difficult once you have the data from the touchscreen : is it a USB or Serial interface ?

    1. Can you access it via a virtual comm port on your system ? (say with ‘screen’ for example: “screen /dev/tty.usbFTDI1234″ [ctrl-a then ctrl-k to quit] ).

    2. Can you open up the casing and inspect what USB chipset it is using ? (FTDI is very common and would make it a breeze).

    You can email me directly with alex at geekorgy — and once we find a workable solution i will post it here aswell.

    cheers

    Alex.

  • Matt says:
    8-06-10

    Hi Alex,

    Thanks for getting back to me!

    Unfortunately, I don’t have the screen yet. There have been too many times where I’ve come up with an idea to do something, gone ahead and bought everything only to stop halfway. Now I’m doing all I can to make sure it’s possible before buying anything.

    I can however give you a link to the screen I’m planning on getting; http://www.mimomonitors.com/products/imo-pivot-touch

    Also, the link to the driver is: iMo Touch/iMo Mini-Monster Touch Touchscreen Driver Intel-based Macs only (Mac OS X 10.4.11 or later)

    Do you have any opinions on any touchscreens that work particularly well with OS X?

    I’m sorry that I don’t really know what I’m on about , but your help is hugely appreciated.

    Thanks,

    Matt

  • Andrew says:
    8-06-10

    Hi Alex
    Thanks so much for posting this. Like others I spent many frustrating hours until I found your helpful comments and code.
    All the best
    Andrew

  • Alex says:
    8-06-10

    hi Andrew,

    glad it helped!

    cheers

  • Melpomene says:
    8-06-10

    Hi, I seem to be unable to import the Quartz.CoreGraphics modules. Is it something I have to add to my PATH?

    I have a lib called CoreGraphics in /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/.

  • Melpomene says:
    8-06-10

    No need to answere, I found that I had to download Quartz and install from here http://pyobjc.sourceforge.net/downloads.html

  • philleb says:
    8-06-10

    Hopefully you can help me:
    I really need a mouse click via. Python(without X11). I thought it should be pretty simple via. ObjC, but not.
    I´ve installed it via easy_install, if I want to run your code this error-code appears:
    python click.py 100 100
    mouse click at 100 , 100 in 0 seconds
    Traceback (most recent call last):
    File "click.py", line 14, in
    from Quartz.CoreGraphics import CGEventCreateMouseEvent
    File "build/bdist.macosx-10.6-intel/egg/Quartz/__init__.py", line 6, in
    File "build/bdist.macosx-10.6-intel/egg/Quartz/CoreGraphics/__init__.py", line 9, in
    File "build/bdist.macosx-10.6-intel/egg/CoreFoundation/__init__.py", line 22, in
    File "build/bdist.macosx-10.6-intel/egg/CoreFoundation/_CFBag.py", line 7, in
    File "build/bdist.macosx-10.6-intel/egg/CoreFoundation/_CFBag.py", line 6, in __bootstrap__
    RuntimeError: Wrong version of PyObjC C API

    I really dont know why I have the wrong PyObjC C API. Maybe you could help me.
    lg

  • dfred says:
    8-06-10

    This script has come in handy quite a bit. Although, I ran into some stumbling blocks, so here are some tips to others who may be looking for a solution:

    If you install Python 2.7 on your Mac OS 10.6 you have a few options:

    A) In the terminal, type “python2.6″ instead of “python” before the path to the script

    OR

    B) You can install PyObjC by doing the following:
    1) Install easy_install from http://pypi.python.org/pypi/setuptools
    2) In the terminal, type “which python” and copy the path up through “2.7″
    3) Then type “easy_install –prefix /Path/To/Python/Version pyobjc==2.3″
    (ie. easy_install –prefix /Library/Frameworks/Python.framework/Versions/2.7 pyobjc==2.3)
    4) Inside the script type “import objc” at the top

    OR

    C) You can reset your python path to the original Mac OS python:
    1) In the terminal, type “defaults write com.apple.versioner.python Version 2.6″

    Hope this helps.

  • dfred says:
    8-06-10

    Also, a quick way to find out the (x,y) coordinates on the screen:

    Command+Shift+4 (The cursor then shows the coordinates)
    Then hit Esc to get out of it.

  • Tony says:
    8-06-10

    FWIW, if you need to do a mouse drag:

    import time
    from Quartz.CoreGraphics import *
    def mouseEvent(type, posx, posy):
    theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)
    CGEventPost(kCGHIDEventTap, theEvent)
    def mousemove(posx,posy):
    mouseEvent(kCGEventMouseMoved, posx,posy);
    def mouseclickdn(posx,posy):
    mouseEvent(kCGEventLeftMouseDown, posx,posy);
    def mouseclickup(posx,posy):
    mouseEvent(kCGEventLeftMouseUp, posx,posy);
    def mousedrag(posx,posy):
    mouseEvent(kCGEventLeftMouseDragged, posx,posy);
    ourEvent = CGEventCreate(None);
    # Save current mouse position
    currentpos=CGEventGetLocation(ourEvent);
    # move mouse to upper left of screen
    mouseclickdn(40, 60);
    # drag icon to new location
    mousedrag(60, 300);
    # release mouse
    mouseclickup(60, 300);
    # necessary delay
    time.sleep(1);
    # return mouse to start positon
    mouseclickdn(int(currentpos.x),int(currentpos.y));

Post a comment