{"id":41,"date":"2015-10-20T22:40:55","date_gmt":"2015-10-21T05:40:55","guid":{"rendered":"http:\/\/www.howardklein.net\/?p=41"},"modified":"2015-10-20T22:40:55","modified_gmt":"2015-10-21T05:40:55","slug":"python-coroutine-options-part-1-generators","status":"publish","type":"post","link":"https:\/\/www.howardklein.net\/?p=41","title":{"rendered":"Python Coroutine Options, Part 1: Generators"},"content":{"rendered":"

As I mentioned\u00a0here<\/a>, coroutines provide a means to express processes occurring over simulated time in a pretty natural way. Python has one built-in mechanism, the generator, which enables coroutine-like functionality.\u00a0 Two other flavors of coroutine, greenlets<\/a>\u00a0and tasklets<\/a>, are implemented by third-party packages; these will be discussed in a later post.<\/p>\n

Simply put, generators are Python functions containing one or more yield<\/span> statements; they were introduced in Python 2.2. I\u2019m not going to attempt to provide either a complete description of or tutorial on generators; there are plenty of those elsewhere on the web, written by folks much more qualified than myself.\u00a0 Let it suffice to say that:<\/p>\n

    \n
  1. A generator is an iterator created by calling the generator function<\/li>\n
  2. The generator function starts executing when the built-in next()<\/span> is invoked on the generator (which can occur implicitly via a loop)<\/li>\n
  3. Any yield<\/span> <\/span>statement within the generator function returns control (and optionally, a value) to the generator\u2019s caller while allowing the caller to later resume execution of the generator function from where the yield<\/span> was invoked.<\/li>\n<\/ol>\n

    This feature allows us to create a discrete event simulator in which processes are coded as generator functions.\u00a0 Any process action which consumes (or might consume) simulated time takes the form, in code, of a yield expression.\u00a0 For example, a simple process that acquires a resource, waits for ten time units, and then releases the resource might look something like this:<\/p>\n

    yield acquire_resource(rsrc1)\r\nyield wait_for(10)\r\nrelease_resource(rsrc1)<\/pre>\n
    This is, in fact, the basic approach taken by\u00a0SimPy<\/a>, an existing open source simulation framework. \u00a0It works, but it does have some limitations.\u00a0 In particular, generators, as initially implemented, were not easily stackful; while a generator can invoke a regular function, that function cannot in turn invoke yield<\/span> in any form while still being a \u201cregular function\u201d \u2013 once we include a yield<\/span> statement, it becomes another generator function, and calling it simply returns a generator (rather than \u201cdoing\u201d something).<\/p>\n
    This restriction is not insignificant from our perspective.\u00a0 It means, for example, that a process implemented through a generator function cannot invoke a wait()<\/span> function that blocks (through a yield<\/span>) completely behind the scenes.\u00a0 It also makes it more difficult for someone coding a complex process to decompose that process into subroutines, as the simulation would fail to execute properly if any of these subroutines attempted to yield<\/span>, either directly or indirectly.<\/p>\n
    This limitation was partially addressed in Python 3.3, which introduced enhanced support for subgenerators*<\/sup> via the yield from<\/span>\u00a0syntax, e.g.<\/p>\n
    yield from subprocess()<\/pre>\n
    where subprocess<\/span> is another generator function \u2013 a subgenerator. Both the generator and the subgenerator will maintain their state between yields, allowing us to create, in effect, pretty-much stackful coroutines. The price we pay is the need to make any function that might block (or call a subroutine that potentially blocks) a generator, and the requirement to invoke that function via a yield<\/span> from statement \u2013 all of which looks somewhat (OK, a lot<\/em><\/strong>) messier than we would like, and is fraught with the potential for bugs that might not make themselves obvious at run time.\u00a0 If you don\u2019t expect your function to be a generator and it is, your program may run without complaint, but it almost certainly won\u2019t run as designed.<\/p>\n
    *<\/sup>Strictly speaking the use of subgenerators was possible prior to Python 3.3, but doing so in a robust way was pretty challenging and generally not recommended.<\/p>\n","protected":false},"excerpt":{"rendered":"
    As I mentioned\u00a0here, coroutines provide a means to express processes occurring over simulated time in a pretty natural way. Python has one built-in mechanism, the generator, which enables coroutine-like functionality.\u00a0 Two other flavors of coroutine, greenlets\u00a0and tasklets, are implemented by third-party packages; these will be discussed in a later post.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[3,5,2],"tags":[],"class_list":["post-41","post","type-post","status-publish","format-standard","hentry","category-modeling","category-python","category-simulation"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/posts\/41","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=41"}],"version-history":[{"count":4,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/posts\/41\/revisions"}],"predecessor-version":[{"id":45,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=\/wp\/v2\/posts\/41\/revisions\/45"}],"wp:attachment":[{"href":"https:\/\/www.howardklein.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=41"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=41"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.howardklein.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=41"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}