To implement parallelism in an application, a developer has three choices:
- Non-blocking APIs with callbacks
- Fiber coroutines
Programming with threads can be appealing at first, however a multithreaded design can be problematic. Subtle programming errors due to timing related issues, multithread lock deadlocks and race conditions can be extraordinarily difficult to detect and diagnose. All too often, they appear in production deployments.
The second approach of using non-blocking APIs with callbacks is simpler to debug. But code quality suffers with the all too common "callback-hell". Relatively simple algorithms become obscure when scattered over cascading callbacks.
A compelling alternative, is to use Fiber coroutines. A fiber coroutine is code that runs with its own stack and cooperatively yields to other fibers when it needs to wait.
Fibers allow programs to overlap waiting for I/O or other events with useful compute tasks. They achieve parallelism without the pain.
Fibers solve the main problem with multi-threaded programming where multiple threads access the same data at the same time and require complex locking to safeguard data integrity. Fibers solve the main problem with non-blocking callbacks by enabling a procedural straight line coding style.
Fibers are not perfect. They will not let you utilize all the CPU cores of a system within one program. But for the use case of embedded device management, this is not a significant concern. Device management applications are generally secondary in purpose to the primary role of the device and consequently should not be monopolizing the CPU cores of the device.
Consider a threaded example:
Now consider the fiber solution:
Since only one segment of code is executing at any one time, there is no possibility of fiber collisions.
When doing I/O, applications can choose to perform blocking or non-blocking I/O. Blocking I/O while being simpler, means the application cannot perform any other functions while waiting for I/O to complete.
Consider an application that needs to perform a REST HTTP request to retrieve some remote data. While this request is waiting several seconds, the application cannot perform any other task as it is blocked waiting for the request to complete.
Non-blocking I/O solves this problem, but creates another: "callback hell".
Consider this pseudo-example:
You can see that callbacks quickly obscure the code's intent.
The alternative Ioto code using fiber coroutine would look like this:
The calls to urlFetch will yield and other fibers will run while waiting for I/O. When the request completes, this fiber is transparently resumed and execution continues.
Fiber-based code is simpler to code, debug and maintain. When converting Ioto from callbacks to fibers, several of our algorithms reduced in lines of code by over 30%.
Fibers in Practice
In practice, you typically don't need to explicitly code fiber yielding or resuming. The Ioto socket APIs are fiber-aware and will do the yielding for you. The rReadSocket and rWriteSocket APIs will block the current fiber as required, but other fibers will continue to run. NOTE: that only one fiber is ever running at a time.
In all Ioto services, including the web server, Url client, MQTT client and AWS services: the async APIs are fiber-aware and will yield and resume automatically.
When using the Ioto fiber coroutines, your main program typically performs little processing before calling rInit to create your first fiber. This fiber can then continue initialization and use the full fiber API.
While you can use many of the "R" runtime APIs from your main program (outside a fiber coroutine), you cannot call rReadSocket, rWriteSocket and rConnectSocket in your main program. It is best practice to call rInit as soon as possible and complete initialization inside a fiber.
Ioto I/O API
Ioto builds fiber support into the lowest layer of the "R" portable runtime. The following APIs support automatic fiber yielding:
These APIs will automatically yield and resume as required.
Furthermore, if you are using TLS, the rConnectSocket API must only be called inside a fiber. This is because the handshaking exchange I/O is performed using fiber read/write primities internally.
Ioto supports a low level fiber API so you can construct your own fiber-enabled primitives.
Use rYieldFiber to yield the CPU and switch to another fiber. You must make alternate arrangements to call rResumeFiber when required.
Use rSpawnFiber to create a new fiber and immediately switch to it. For example:
Integrating with External Services
But what should you do if you need to invoke an external service that will block?
You have two alternatives:
- Use Non-Blocking APIs
- Use threads
Ioto provides a flexible centralized eventing and waiting mechanism that can support any service that provides a select() compatible file descriptor.
If the external service has a non-blocking API and provides a file descriptor that is compatible with select or epoll, you can use the Ioto runtime wait APIs to be signaled when the external service is complete.
To wait for I/O on a file descriptor, call rAllocWait to create a wait object and rSetWaitHandler to nominate an event function to invoke.
The nominated function will be run on a fiber coroutine when I/O on the file descriptor (fd) is ready.
The other option is to create a thread. However you must take care to properly yield the fiber first. The runtime provides a convenient rSpawnThread API that will do this for you. It will create a thread, yield the current fiber and then invoke your threadMain. When your threadMain exits and returns a result, it will automatically resume the fiber and pass the result as the return value from rSpawnThread.
It is generally not safe to call any runtime APIs within a foreign thread except for: rStartEvent, rResumeFiber, rStartFiber and the thread locking primitives.
If you need to invoke a runtime API from within a foreign thread, you should call rStartEvent API to safely invoke a function that executes on a runtime fiber.
Manual Yield and Resume
Though unlikely, you may have a need to manually create fibers and yield and resume explicitly.
The APIs for this are: rAllocFiber, rYieldFiber and rResumeFiber.
See the Runtime API for more details.