CloudI 2.0.7

CloudI is an open-source private cloud computing framework for efficient, secure, and internal data processing. CloudI provides scaling for previously unscalable source code with efficient fault-tolerant execution of ATS, C/C++, Erlang/Elixir, Go, Haskell, Java, JavaScript/node.js, OCaml, Perl, PHP, Python, Ruby, and Rust services. The bare essentials for efficient fault-tolerant processing on a cloud!

Tags erlang elixir c++ c golang haskell java javascript ocaml perl php python ruby unix cross-platform distributed-computing
License MITL
State stable

Recent Releases

2.0.727 Oct 2023 22:29 minor bugfix: * Python/C CloudI API compilation and reliability was improved * C/C++ CloudI API reliability was improved by removing use of nongnu-libunwind for the C++ backtrace information * Added the cloudi_service_health_check tcp_test argument functions tcp_test_http and tcp_test_https to be provided as closure data: cloudi_service_health_check, tcp_test_https, get, "/", 200 * Added the cloudi_service_router http_redirect_health argument for using cloudi_service_health_check results to alter the HTTP redirect response provided during a remote outage * Erlang/OTP 26.1 support was added * Haskell GHC 9.8.1 and 9.6.3 support was added * Added the cloudi_service_validate_config service for service configuration validation based on all configuration values (for logging possible mistakes during dynamic use) * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.610 Jul 2023 20:03 minor feature: * Added the Rust CloudI API (rustc = 1.66.1) * Added the bind service configuration option for binding CloudI service execution to logical processors (supported by the ATS, C/C++, Erlang, Python/C CloudI API when the Erlang VM was started with the +sbt command-line argument) * Added static CloudI API functions for use before initialization: timeout_initialize, timeout_terminate, process_index, process_count_max, process_count_min * Added cloudi_service_cron support for random ranges with the (tilde) character based on OpenBSD cron * Added the cloudi_service_shell interactive argument which allows accessing an interpreter with CloudI service requests * Added the critical service configuration option for stopping the CloudI node when a critical service failed after MaxR restarts * Erlang/OTP 26 support was added * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.513 Oct 2022 08:03 minor bugfix: * backwards incompatible changes: * external services previously had the service configuration option fatal_exceptions functionality set to false * Added the fatal_exceptions service configuration option for whether an uncaught exception should cause a service restart or a null response (defaults to true) * Added the fatal_timeout service configuration option for whether a service request timeout causes a service restart to ensure the service request is terminating (defaults to false) * Added the fatal_timeout_delay service configuration option for providing an additional offset on the service request timeout * All service configuration time values now accept the "period" format to represent a time period as a 2-tuple like: 1, minute or 5, hours (service configuration is simplified in the log output using this format and it is defined in the cloudi_service_api module types) * Added the cloudi_service_health_check service for monitoring the availability of any hosts based on a TCP port or ping response * Added a visual ASCII representation of outages to the CloudI Service API services_status and nodes_status response data (cloudi_service_health_check also has a hosts. erl,json response) * Added the CloudI Service API logging_set file_sync configuration option to delay calling fdatasync after a logging file line is written (defaults to 0 milliseconds) * Switched CloudI logger output from usage of ?MODULE to ?FILE for log output from header files * cloudi_service_http_cowboy now automatically load-balances among Erlang processes when count_process 1 (on Linux and FreeBSD) * Updated load test results with commands used at https://github.com/CloudI/CloudI/tree/develop/src/lib/cloudi_service_request_rate/results/results_v2_0_4#readme * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.405 Dec 2021 10:43 major bugfix: * backwards incompatible changes: * All service name and service name patterns must be UTF-8 * The ATS CloudI API subscribe function now uses the implement template idiom to simplify use * Improved the C/C++ CloudI API backtrace information * Added the cloudi_service_send service for simpler automation * Fixed the service configuration options aspects_suspend and aspects_resume * OCaml 4.13 support was added * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.228 May 2021 07:18 major bugfix: * Added the ATS CloudI API (ATS2/Postiats = 0.3.13) * Added more information to the CloudI Service API functions services_status, nodes_status, logging_status and code_status * Added the overload queue_mode to the CloudI logger to prevent node logging redirects from causing excessive memory consumption * Updated cloudi_service_http_cowboy to use cowboy 2.9.0 * Added cache replacement algorithms to cloudi_service_filesystem (replace argument may be lfuda, lfuda_gdsf or lru) * Added queue dependencies to cloudi_service_api_batch * Added more descriptive C/C++ CloudI API termination handler output * Erlang/OTP 24 support was added * Added hackney support to cloudi_service_http_client * Added the service configuration options aspects_suspend and aspects_resume for handling service suspend state changes * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.128 Nov 2020 01:45 major bugfix: * backwards incompatible changes: * CloudI API external service request exception handling will now cause a service restart for unrecoverable exceptions and assertions (e.g., Error in Java) * CloudI Service API code_path_add, code_path_remove and code_path uses separate storage of paths (separate from Erlang) * FatalError exception added to external service CloudI APIs to cause a service restart for unrecoverable service errors * Added the CloudI Service API functions logging_status and logging_status_reset for tracking file output details * 'code' configuration file section was added to simplify the loading of Erlang source code * cloudi_service_funnel was added for providing coordination among duplicate service requests that receive a single response * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
2.0.005 Jun 2020 01:57 major feature: * backwards incompatible changes: * CloudI API initialization has an optional terminate_return_value parameter (defaults to true for past functionality) that causes a terminate exception to be used if set to false, in C++/OCaml/Haskell * info_key_value_parse CloudI API function is now static in Go/JavaScript/Perl/PHP/Python/Ruby * Fixed CloudI Service API nodes_set (nodes configuration) use of connect and listen for hidden node connections (isolates one CloudI cluster from another by not creating a fully-connected network topology when connect == hidden) * Added the service configuration option restart_all to restart all service processes when one fails * Added the CloudI Service API services_suspend and services_resume functions to pause the processing of incoming service messages * Added suspend-related information to the CloudI Service API services_status output * Added C/C++ CloudI API functions for automatically freeing memory passed to return or forward * Erlang/OTP 23 support was added * Python 3.8 support was added * Haskell GHC 8.10.1 support was added * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.8.029 Nov 2019 20:35 minor feature: * backwards compatibility difference: * ACL strings do not get a "*" automatically appended on non-patterns, so exact strings may be used as an ACL string * Unicode support was added for external service configuration (file_path, args, env) and any other string or function use * CloudI API now has a shutdown function for external services (internal services always had the shutdown return value) * Fixed CloudI Service API services_add failure with multiple service configurations to handle data properly * Fixed service configuration option count_process_dynamic so it doesn't influence a pending service termination * Erlang services were added: * cloudi_service_api_batch executes services with batch queues * cloudi_service_cron sends service requests for cron expressions * cloudi_service_shell executes service request shell commands * Java service cloudi_service_htmlunit was created (see https://github.com/CloudI/cloudi_service_htmlunit/) * Haskell GHC 8.8.1 support was added * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.7.519 Dec 2018 04:01 minor bugfix: * Added CloudI Service API nodes_status function to provide current uptime, availability and cost information for any CloudI node (using Erlang monotonic time) * Now both "*" and "?" are wildcard characters in service name patterns that consume one or more characters, with "?" never matching the next character in the string (i.e. "/?/" matches "/a/" but never "/a/b/" while "/*/" will match either) * cloudi_service_http_cowboy was renamed to be cloudi_service_http_cowboy1 * cloudi_service_router ssh server throughput was improved * All external dependencies were updated * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.7.416 Sep 2018 23:23 minor bugfix: * Added CloudI Service API services_status function to provide current uptime and availability information for any CloudI service (using Erlang monotonic time) * Added CloudI Service API code_status function to provide information about the installation and any runtime changes of the CloudI service files on the filesystem * ZeroMQ support was removed and SSH connectivity was added to cloudi_service_router as a better alternative * cloudi_crdt had small additions and an important bugfix * Added JSON support to cloudi_service_api_requests * Fixed Javascript CloudI API service termination (no delay now) * Added support in Python, Python/C and Java CloudI APIs for non-member (static) functions (passed to the subscribe function) * Erlang/OTP 21.x compatibility * Haskell CloudI API dependencies were updated * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.7.328 Feb 2018 01:29 minor bugfix: * cloudi_crdt was added to provide an easy way to have replicated state among internal CloudI service processes (implementing a POLog CRDT with an Erlang map, see the Erlang count integration test for example use) * All cloudi_core and internal CloudI service time tracking has switched to monotonic time where it is advantageous (e.g., CloudI service restart times, cloudi_queue, etc.) * A bug preventing dest_list_deny and dest_list_allow from being used for cloudi_service_http_cowboy, cloudi_service_http_elli and cloudi_service_tcp was fixed (cloudi_service_children.hrl bug) * cloudi_queue will now perform a new service name lookup upon a retry if the pattern_pid() was not provided explicitly * The file_size_limit argument was added to cloudi_service_queue with the default value of 128MB * Javascript CloudI API had a busy-wait bug causing excessive CPU consumption which has been fixed * Java 9 support was added * OCaml = 4.06 support was fixed by including num as a compile-time dependency * CloudI execution now uses a default umask of 0027 which may be set with CLOUDI_UMASK (before the configure script is called) * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.7.213 Sep 2017 20:52 minor bugfix: * Service configuration option request_timeout_adjustment was modified to use Erlang monotonic time so timeout adjustments are not impacted by any OS time changes (now all CloudI service request timeout handling is using monotonic time) * Added CloudI Service API logging_set configuration option log_time_offset to log when the Erlang VM updates its time-offset with the time-offset change logged in nanoseconds (the offset is logged exactly when logging timestamps are changed) * configure script now accepts environment variables to set release configuration values: CLOUDI_ARG_NODE_NAME_SHORT, CLOUDI_ARG_NODE_NAME_LONG, CLOUDI_ARG_NODE_COOKIE, CLOUDI_PID_FILE (can be used to avoid manual vm.args modifications) * Many deployment changes to make CloudI use robust: * The release script is safe for root use (external environment variables are blocked, functionality is locked down, manages a pid file correctly, etc.) * Execution now occurs within the logging directory so any crash dump output is put there automatically * cloudi_service_null is an Erlang service added to provide a simple way of consuming CloudI service requests (during testing and/or to handle invalid URL service names) * cloudi_service_request_rate was added to the CloudI release to provide an easy way of testing max throughput * cloudi_service_http_cowboy and cloudi_service_http_elli now use an update_delay argument to apply a change to timeout_async and/or timeout_sync performed with the CloudI Service API services_update * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.7.108 Jun 2017 04:08 minor bugfix: * All CloudI source code is under a MIT license instead of a BSD-style license to make reuse simpler and to be GPL-compatible * The chroot external service configuration option was added (http://cloudi.org/api.html#2_services_add_config_opts) * The order external service configuration options are applied changed from owner, limit, nice, directory to chroot, limit, nice, owner, directory which means that the user/group responsible for running CloudI influences the permissions for chroot, limit, nice (and cgroup). This order will not need to change in the future. * Erlang/OTP 20.0 compilation/runtime works * backwards compatibility difference: * cpg (CloudI Process Groups) source code was changed to minimize distributed Erlang communication (and local messaging) without attempting to remain compatible with cpg versions before 1.7.1 (i.e., prevent CloudI 1.7.1 nodes from communicating with older nodes by using different cookie settings or other means). * Using the CloudI API functions send_sync or recv_async during the service's initialization previously returned an error, but will now cause the service to crash, to pursue fail-fast operation and to keep the effect the same for both internal and external services.
1.7.029 Mar 2017 23:10 minor feature: * backwards compatibility difference: * The C CloudI API (not the C++ CloudI API) had a small change to facilitate the use of thread local data with the callback function arguments order changing to match the other programming languages that lack (or avoid) state objects (C, Erlang, Go, Haskell, OCaml) * The Go CloudI API was added (requires the configure argument --enable-go-support and Go (gc) = 1.6) * The Haskell CloudI API was added (requires the configure argument --enable-haskell-support and GHC = 7.10.3, cabal-install = 1.22) * The OCaml CloudI API was added (requires the configure argument --enable-ocaml-support and OCaml = 4.03.0) * The CloudI Service API function logging_stdout_set was added to allow all CloudI logging to be sent to stdout * The count integration test was added to show how thread local state works in all supported programming languages (http://cloudi.org/tutorials.html#cloudi_examples). (see https://github.com/CloudI/CloudI/blob/v1.7.0/src/NEWS for more details)
1.6.012 Jan 2017 00:49 minor bugfix: * backwards compatibility difference: * Functions deprecated in the 1.3.3 CloudI release have been removed * Usage of the 'immediate' atom for the timeout value with the cloudi_service Erlang module and the cloudi Erlang module is deprecated, use the 'limit_min' atom instead for the same value * The minimum allowable values for service configuration (both adding a new service and updating an existing service) of timeout_async and timeout_sync has changed from 101 milliseconds to 499 milliseconds to be consistent with the value that 'limit_min' represents * The service configuration options request_timeout_immediate_max and response_timeout_immediate_max had their meaning change to rely on a greater than or equal to comparison instead of a greater than comparison, though the same default behavior is preserved with this change (http://cloudi.org/api.html#2_services_add_config_opts) * Now cloudi_core doesn't use any port drivers or NIFs except what is provided within Erlang/OTP, and process dictionary use is minimal * The Elixir module CloudILogger is now included in the cloudi_core Erlang application to simplify usage of CloudI in Elixir source code * Bugs related to CloudI logging syslog integration have been fixed and more options for syslog use have been added (http://cloudi.org/api.html#2_logging_syslog_set has more information) * The restart_delay service configuration option was added (http://cloudi.org/api.html#2_services_add_config_opts_restart_delay) * CloudI service restarts have been made synchronous to ensure all termination has completed before initialization of new processes begins (to improve reliable management of global state during a restart) (see https://github.com/CloudI/CloudI/blob/v1.6.0/src/NEWS for more details)
1.5.408 Oct 2016 01:51 major bugfix: * An important bugfix was added for internal services that use both CloudI service requests and info messages at high request rates with the duo_mode service configuration option set to true (service state returned from the info message handling wasn't always saved previously under these conditions) * Request rate testing was done both with and without the persistence of service request data (using cloudi_service_queue in destination mode) as described at http://cloudi.org/faq.html#5_LoadTesting * The 'nice' and 'cgroup' service configuration options were added for external services * The logging_set CloudI Service API function was added to allow multiple logging modifications to occur dynamically (similar to nodes_set for CloudI nodes configuration) * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.5.316 Sep 2016 02:02 major feature: * services_update is a new addition to the CloudI Service API for updating any service without downtime, i.e., module hot-code reloading with internal services and OS process restarts with external services while CloudI ensures service requests are not lost and service request handlers are not interrupted in any of the service processes or threads (see http://cloudi.org/api.html#2_services_update for more information) * A Java user tutorial has been added based on the work of Bruce Kissinger (http://cloudi.org/tutorial_java.html) * The Java CloudI API can now use method references with Java 8 * New service configuration options were added or improved: 'directory', 'dispatcher_pid_options', 'init_pid_options', 'request_pid_options', 'info_pid_options' (http://cloudi.org/api.html#2_services_add has more information) * Many of the database services were removed with their dependencies, from the main repository due to service development demonstrating that failures remain isolated when using a database client locally within a service (database services can help scale usage and make sure it is fault-tolerant, but only after the service business logic is in a stable state and excluding these services will help guide developers) * cloudi_service_monitoring was improved: * Any Erlang process can create or update metrics now * cloudi.log output can be monitored with aspects_log_before or aspects_log_after * All CloudI service related metrics now have a static metric name, identified with TYPE.FILENAME.INDEX instead of the service ID as described in the ChangeLog file * Erlang internal services now have two optional behaviour functions (some services choose to use only one of the functions): * cloudi_service_handle_request/11 * cloudi_service_handle_info/3 * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.5.125 Dec 2015 00:14 minor bugfix: * cloudi_service_monitoring is a new CloudI service that provides metrics based on the Erlang VM and all the running CloudI services with exometer and folsom * cloudi_service_http_rest is a new CloudI service to simplify the creation of HTTP REST API handlers * CloudI service efficiency was improved by switching to maps with 18.x * CloudI now utilizes the new time warp functionality while retaining globally unique transaction ids * The CloudI logging output now has an entry for the function/arity info which is captured in Elixir and can be provided in Erlang (until EEP45) * Javascript CloudI API now works properly with node.js version 0.12.1 * Loadtests were ran (http://cloudi.org/faq.html#5_LoadTesting) * Bugs were fixed and other improvements were added (see the ChangeLog for more detail)
1.5.013 Apr 2015 07:40 minor bugfix: * backwards compatibility difference: * cloudi module (Erlang/Elixir-only usage) now returns a 2 element tuple with the 1st element as the result (original return value) and the 2nd element as the modified Context data (or the Dispatcher pid if a Dispatcher pid was provided instead of a Context) * Was a necessary change to keep the transaction id time as strictly monotonically increasing with the changes in Erlang 18.x that deprecate erlang:now/0 * Allows a lazy destination refresh to update the service name lookup data so the developer doesn't need to handle the update manually * The change impacted the return value of internal service external interface functions * The service application was changed to depend only on the cloudi_service module instead of the cloudi module to make its usage clearer and avoid Dispatcher pid value handling * Despite the return value changes, only the minor version number of CloudI was incremented due to the interfaces being outside the main documentation * External services can now set OS process resource limits with the 'limit' service configuration option (http://cloudi.org/api.html#2_services_add_config_opts_limit) * A few new CloudI data structures: cloudi_queue, cloudi_future, cloudi_service_future (cloudi_queue can handle service request validation, failures, and retries with in-memory internal service data) (cloudi_future and cloudi_service_future provide Futures for Erlang processes (sending to CloudI services) and internal CloudI services, respectively) * cloudi_service_validate is a new CloudI service for validating service requests with the requests using the service as a proxy (similar functionality was added to cloudi_service_quorum and cloudi_service_router, though cloudi_service_router is forwarding the service request instead of sending it) *
1.4.020 Jan 2015 22:14 cleanup: December 20, 2014 Version 1.4.0 released. * JavaScript/node.js, PHP and Perl are now supported programming languages with their own CloudI API implementations * A Quick Start now has intros for all supported programming languages (http://cloudi.org/#quick_start) * Loadtests were ran (http://cloudi.org/faq.html#5_LoadTesting) * Various bugs were fixed (see the ChangeLog for more detail)
1.3.301 Oct 2014 19:31 major feature: September 30, 2014 Version 1.3.3 (beta) released. * backwards compatibility difference: * Some minor functions were deprecated in the cloudi_service module and the source code was moved to separate modules (cloudi_environment, cloudi_service_name, cloudi_request, cloudi_request_info, cloudi_key_value) * The CloudI Service API had some functions renamed with the older name deprecated (loglevel_set became logging_level_set and log_redirect became logging_redirect_set) * In version 1.4.0 of CloudI the cloudi_service behaviour will have a Timeout parameter in both the cloudi_service_init callback and the cloudi_service_terminate callback (if you want to start using it now, make sure cloudi_core is compiled with the CLOUDI_SERVICE_NEW define, a CLOUDI_SERVICE_OLD macro will exist in version 1.4.0 to avoid the change so older services can run without modification) * cloudi_service_api_requests now provides simpler URL paths for the CloudI Service API (while continuing to support the older URL paths) (https://github.com/CloudI/CloudI/tree/master/src/service_api#api) * Python 3 usage requires that the request_info, request, response_info, and response data be bytes * Elixir is now supported officially and has an example at https://github.com/CloudI/CloudI/tree/master/examples/Elixir.HelloWorld1 * Python 3 is now supported by the Python CloudI API (both the cloudi module and the cloudi_c module) * queue_size is a service configuration option added so any service may limit their incoming service request queue by an amount of memory (in kB) (queue_limit provides a count limit and both can be used together) * Support for Aspects (from AOP, Aspect-Oriented Programming) were added as service configuration options (aspects_init_after, aspects_request_before, aspects_request_after, aspects_info_before, aspects_inf
1.3.227 Jul 2014 08:19 major feature: May 31, 2014 Version 1.3.2 (beta) released. * Fixes for Erlang/OTP 17.0 compilation * Improved node auto-discovery functionality and configuration * Add cloudi_service_api:nodes_set/2 for dynamically configuring CloudI nodes configuration * EC2 node discovery now works as expected, selecting based on tags and/or security groups * Allow node connections to be hidden for more complex network topologies (see http://cloudi.org/api.html#2_nodes_set) * Add cloudi_service_queue for persistent queuing of service requests with CT tests * Add cloudi_service_filesystem file update notification service requests (see notify_one and notify_all configuration) * The cloudi_service_http_cowboy service has many different additions: * Added websocket_protocol for creating a mapping between a protocol id and the TransId of websocket service requests to avoid contention * Added websocket_subscriptions for adding additional websocket connection subscriptions for receiving service requests, conditional on the connection URL (to simplify messaging to groups of websockets) * websocket_connect and websocket_disconnect can be set to be either async or sync service requests * use_websockets == exclusively for only using websockets * Add service name pattern support to cloudi_service_router (to map from a service name pattern to a new service name destination) * The cloudi_service_db_pgsql now provides either the https://github.com/semiocast/pgsql or https://github.com/wg/epgsql driver along with a separate configuration option for providing common output * Log output now reflects the default (tuple) service configuration format * Various bugfixes, small additions, and documentation improvements