internals:extensions

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
internals:extensions [2013/04/09 16:12] – created jpauliinternals:extensions [2017/09/22 13:28] (current) – external edit 127.0.0.1
Line 11: Line 11:
  
 Both extension kinds share lots of stuff. The difference between both types is mainly in hooks they register into the Engine. Both extension kinds share lots of stuff. The difference between both types is mainly in hooks they register into the Engine.
 +Remember that, despite it is very uncommon, an extension can be both a PHP extension and a Zend extension at the same time. Xdebug is a good example.
  
-All extensions may be built statically or dynamically. +Here are the PHP extension structures : 
-When built statically, the extension is loaded earlier than when its built dynamically and loaded throught php.ini.+<code c> 
 +typedef struct _zend_module_entry zend_module_entry; 
 +typedef struct _zend_module_dep zend_module_dep; 
 + 
 +struct _zend_module_entry { 
 + unsigned short size; 
 + unsigned int zend_api; 
 + unsigned char zend_debug; 
 + unsigned char zts; 
 + const struct _zend_ini_entry *ini_entry; 
 + const struct _zend_module_dep *deps; 
 + const char *name; 
 + const struct _zend_function_entry *functions; 
 + int (*module_startup_func)(INIT_FUNC_ARGS); 
 + int (*module_shutdown_func)(SHUTDOWN_FUNC_ARGS); 
 + int (*request_startup_func)(INIT_FUNC_ARGS); 
 + int (*request_shutdown_func)(SHUTDOWN_FUNC_ARGS); 
 + void (*info_func)(ZEND_MODULE_INFO_FUNC_ARGS); 
 + const char *version; 
 + size_t globals_size; 
 +#ifdef ZTS 
 + ts_rsrc_id* globals_id_ptr; 
 +#else 
 + void* globals_ptr; 
 +#endif 
 + void (*globals_ctor)(void *global TSRMLS_DC); 
 + void (*globals_dtor)(void *global TSRMLS_DC); 
 + int (*post_deactivate_func)(void); 
 + int module_started; 
 + unsigned char type; 
 + void *handle; 
 + int module_number; 
 + const char *build_id; 
 +}; 
 + 
 +struct _zend_module_dep { 
 + const char *name; /* module name */ 
 + const char *rel; /* version relationship: NULL (exists), lt|le|eq|ge|gt (to given version) */ 
 + const char *version; /* version */ 
 + unsigned char type; /* dependency type */ 
 +}; 
 +</code> 
 +And here are the Zend extension structures : 
 +<code c> 
 +/* Typedef's for zend_extension function pointers */ 
 +typedef int (*startup_func_t)(zend_extension *extension); 
 +typedef void (*shutdown_func_t)(zend_extension *extension); 
 +typedef void (*activate_func_t)(void); 
 +typedef void (*deactivate_func_t)(void); 
 + 
 +typedef void (*message_handler_func_t)(int message, void *arg); 
 + 
 +typedef void (*op_array_handler_func_t)(zend_op_array *op_array); 
 + 
 +typedef void (*statement_handler_func_t)(zend_op_array *op_array); 
 +typedef void (*fcall_begin_handler_func_t)(zend_op_array *op_array); 
 +typedef void (*fcall_end_handler_func_t)(zend_op_array *op_array); 
 + 
 +typedef void (*op_array_ctor_func_t)(zend_op_array *op_array); 
 +typedef void (*op_array_dtor_func_t)(zend_op_array *op_array); 
 + 
 +typedef struct _zend_extension { 
 + char *name; 
 + char *version; 
 + char *author; 
 + char *URL; 
 + char *copyright; 
 + 
 + startup_func_t startup; 
 + shutdown_func_t shutdown; 
 + activate_func_t activate; 
 + deactivate_func_t deactivate; 
 + 
 + message_handler_func_t message_handler; 
 + 
 + op_array_handler_func_t op_array_handler; 
 + 
 + statement_handler_func_t statement_handler; 
 + fcall_begin_handler_func_t fcall_begin_handler; 
 + fcall_end_handler_func_t fcall_end_handler; 
 + 
 + op_array_ctor_func_t op_array_ctor; 
 + op_array_dtor_func_t op_array_dtor; 
 + 
 + int (*api_no_check)(int api_no); 
 + int (*build_id_check)(const char* build_id); 
 + void *reserved3; 
 + void *reserved4; 
 + void *reserved5; 
 + void *reserved6; 
 + void *reserved7; 
 + void *reserved8; 
 + 
 + DL_HANDLE handle; 
 + int resource_number; 
 +} zend_extension; 
 + 
 +typedef struct _zend_extension_version_info { 
 + int zend_extension_api_no; 
 + char *build_id; 
 +} zend_extension_version_info; 
 +</code> 
 + 
 +===== Statically compiled extensions ===== 
 + 
 +Lots of extensions may be built statically or dynamically. Some require one or the other compilation mode, but they are uncommon
 +When built statically, the extension is loaded **earlier** than when it'built dynamically (loaded throught php.ini).
 So, the first thing to remember is that an extension may behave differently if it is loaded statically against dynamically, at least if the maintainer has not taken care of such a behavior. So, the first thing to remember is that an extension may behave differently if it is loaded statically against dynamically, at least if the maintainer has not taken care of such a behavior.
 +
 +When you prepare your sources, part of the prepare work is to write a file ''main/internal_functions.c''
 +This files contains an array with the pointers to all the to-be-statically-compiled extensions. Here is an example :
 +<code c>
 +static zend_module_entry *php_builtin_extensions[] = {
 + phpext_date_ptr,
 + phpext_ereg_ptr,
 + phpext_pcre_ptr,
 + phpext_foo_ptr,
 + phpext_reflection_ptr,
 + phpext_spl_ptr,
 + phpext_standard_ptr,
 +};
 +</code>
 +
 +Extensions such as ''standard'' are mandatory, they are always compiled and built statically. This is because further dynamically loaded extensions may depend on stuff provided by standard, this happens sometimes as standard provides lots of functionality.
 +**Only PHP extensions may be built statically, Zend extensions may not. Extensions which are both PHP and Zend, may be built statically**
 +
 +The first thing PHP does when it loads, is to get those pointers out, and load those extensions.
 +The pointers are pointers to the ''zend_module_entry'', so they are PHP extensions.
 +
 +Example:
 +<code c>
 +#define phpext_standard_ptr &basic_functions_module
 +
 +zend_module_entry basic_functions_module = { /* {{{ */
 + STANDARD_MODULE_HEADER_EX,
 + NULL,
 + standard_deps,
 + "standard", /* extension name */
 + basic_functions, /* function list */
 + PHP_MINIT(basic), /* process startup */
 + PHP_MSHUTDOWN(basic), /* process shutdown */
 + PHP_RINIT(basic), /* request startup */
 + PHP_RSHUTDOWN(basic), /* request shutdown */
 + PHP_MINFO(basic), /* extension info */
 + PHP_VERSION, /* extension version */
 + STANDARD_MODULE_PROPERTIES
 +};
 +</code>
 +
 +To register a new PHP extension, the engine calls for ''zend_register_module_ex(zend_module_entry *module)''.
 +This does few things :
 +  * Checks for extension dependencies, but only against conflicts, so **it does not load any other extension** than the one it's been called with
 +  * Checks if the extension has already been registered, if it is the case, warns
 +  * Registers the PHP extension functions into the global function table, calling ''zend_register_functions(module->functions)''
 +
 +===== Parsing the ini file to get more extensions =====
 +
 +''php_init_config()'' is called, it parses the different ini files (mainly php.ini) and looks for the tokens "extension=" and "zend_extension=".
 +It then adds the names of the extensions to two different lists, one for PHP exts : extension_lists.functions, and one for Zend exts : extension_lists.engine. Both are zend_llist (Linked List) type.
 +You can find the code by looking for the function ''php_ini_parser_cb()''
 +
 +  Before PHP5.5, Zend extensions needed to be provided the full path to the extension (ini token "zend_extension="), such as /usr/lib/php/foo.so, whereas PHP extensions (using the ini token "extension=") just needed the file name and used the "extension_dir" ini directive to build the full path to load the extension.
 +  As of 5.5, this in not the case any more and both PHP and Zend extensions are loaded providing only the file name, not the full path.
 +  
 +At this stage, PHP and Zend extensions have been parsed from ini files, but nothing more has been done.
 +
 +===== Registering additional extensions from the SAPI module =====
 +Again before loading/registering anything from ini files, any SAPI may call ''php_module_startup()'' passing it an array of PHP extensions to load (pointer to zend_module_entry*)
 +Those get registered here, the same way statically compiled got registered just few steps before, to recall :
 +  * Checks for extension dependencies, but only against conflicts, so **it does not load any other extension** than the one it's been called with
 +  * Checks if the extension has already been registered, if it is the case, warns
 +  * Registers the PHP extension functions into the global function table, calling ''zend_register_functions(module->functions)''
 +
 +For example, the Apache SAPI (apxs2), registers a PHP extension that provides functions related to Apache. Each SAPI may register one or more PHP extension providing additional functionality for itself. Apache is a good example.
 +  The CLI SAPI doesn't register any extension
 +  
 +===== Loading the ini parsed extensions =====
 +
 +Here comes time where PHP will effectively **load** Zend and PHP extensions, calling some hooks onto them. ''php_ini_register_extensions()'' is called and does this job.
 +First thing to know : **Zend extensions are loaded before PHP extensions**
 +
 +==== Loading Zend extensions ====
 +''zend_load_extension()'' is called, the shared object is dlopen()'ed and two symbols are looked for :
 +  * zend_extension_entry
 +  * extension_version_info
 +
 +  If those symbols are missing, the Zend extension will fail to load
 +
 +Then API compatibility is checked from ''extension_version_info'' against current running PHP. Zend extension's ''api_no_check()'' may be called if mismatch.
 +After that, buildid is checked. Buildid is a combination of the previously checked API number, debug flag and thread safety flag (ZTS). Other components may be used for buildid check. Zend extension's ''build_id_check()'' function may be called as well, though it's pretty uncommon.
 +
 +   Usually, this system just checks that the extension contains code that will be loadable and understandable by the PHP internal code actually loading it. For example, an extension built with ZTS activated will hit those checks and won't be loaded on a PHP running without ZTS.
 +   
 +After those checks, the Zend extensions get registered. ''zend_register_extension()'' is called to do this job.
 +A message is dispatched to all previously registered Zend extension to make them know a new Zend extension is being registered. Then, ''message_handler()'' is called on all already registered Zend extensions, and is given as argument the pointer to the being-registered extension.
 +This system has been designed so that Zend extensions may know each other, and eventually fail loading if they guess they are not compatible with each other.
 +
 +   Remember that there is no such thing like "dependencies" with Zend extensions. Only PHP extensions have a built-in dependency system with features we'll talk about later
 +   
 +Finally, the Zend extension is registered, it is then added to an exported global register storing Zend extensions pointers : ''zend_extensions (zend_llist type)''
 +
 +==== Loading PHP extensions ====
 +After Zend extensions got registered, comes the PHP extensions turn. '' php_load_extension()'' does this job for each of them.
 +So, I repeat again : to register a new PHP extension, the engine calls for ''zend_register_module_ex(zend_module_entry *module)''.
 +This does few things :
 +  * Checks for extension dependencies, but only against conflicts, so **it does not load any other extension** than the one it's been called with
 +  * Checks if the extension has already been registered, if it is the case, warns
 +  * Registers the PHP extension functions into the global function table, calling ''zend_register_functions(module->functions)''
 +
 +   At this stage, the extensions order is not relevant. Extensions are simply registered, nothing more will happen here
 +
 +PHP extensions registration is quite the same as Zend extensions registration : dlopen() the shared object, and checks for compatibility.
 +PHP extensions must export a ''get_module'' symbol. Note that this is different from Zend extensions. Zend extensions required that two symbols exist in the .so file, and those two symbols were structures. PHP extensions on their side only require one symbol to be declared, ''get_module'', and it will get casted as a function returning the ''zend_module_entry'' pointer.
 +
 +The ''get_module'' symbol is effectively exported when you compile your extension. Something like that in your extension main file :
 +<code c>
 +#define ZEND_GET_MODULE(name) \
 +    BEGIN_EXTERN_C()\
 + ZEND_DLEXPORT zend_module_entry *get_module(void) { return &name##_module_entry; }\
 +    END_EXTERN_C()
 +
 +#ifdef COMPILE_DL_FOO
 +ZEND_GET_MODULE(foo) 
 +#endif
 +</code>
 +
 +Then, the two fields ''zend_api'' and ''build_id'' from the freshly loaded ''zend_module_entry'' are checked against PHP internal API. Very similar to Zend extensions checks, except that at this stage, a PHP extension cannot check for other PHP extensions presence, like Zend extensions loading mechanism would allow. Dependency checking is done later, wait :-)
 +
 +After that, the loading mechanism fills in the ''module_number'' field from the PHP extension with just an integer that gets incremented by one at every PHP extension registration/load.
 +Now the PHP extension is going to be registered into a shared global variable, called ''module_registry'', being of type HashTable.
 +The registration step will perform two checks against the PHP extension being registered :
 +  * If the PHP extension is already present in the registry, warning then failure at registration
 +  * PHP extension dependencies **conflicts only** are checked
 +
 +The system checks for the eventually attached ''zend_module_deps'' to the ''zend_module_entry''. This structure basically allows a PHP extension to declare extensions it depends on, and dependencies have types :
 +  * A PHP extension can tell the system that it is not compatible with another already registered extension, so it will fail loading itself into the engine
 +  * A PHP extension can tell the system that it requires another PHP extension to be loaded before itself
 +
 +Here, **only conflicts are checked**, and as there is no particular registration order, a PHP extension A could declare being in conflict with B, but B could be loaded after A, hence the check won't work and you'll probably get some trouble.
 +When dealing with conflicts, it is better to sort the extensions in the php.ini file as a FIFO.
 +**PHP extensions will be registered in the exact same order they've been declared in php.ini**
 +
 +==== Activating PHP extensions ====
 +After registration, comes activation, and here is the step where the engine will effectively sort the PHP extensions in an order that make the dependencies be activated in a specific order.
 +   PHP extensions are activated before Zend extensions
 +
 +''zend_startup_modules()'' does the job of activating PHP extensions. It starts by sorting them in the ''module_registry'' Hashtable calling a sorting callback. This sort process will check for dependencies requirement, and sort the registry in a way that dependency requirements are activated first.
 +Then comes the "real" activation :  ''zend_startup_module_ex()'' is called on the freshly sorted module_registry.
 +It checks the extension field ''module_started'' , just a flag not to activate the same extension more than once. It then checks dependencies against requirements. If an extension requires another to be registered before itself and it's not the case, then an error will show up.
 +   Note that conflicts requirements have already been checked against, at extension loading (see last chapter)
 +After that, the extension globals are registered (call to ''globals_ctor()'' on the extension) and ''module_startup_func()'' is called on the extension, this is known as being the "MINIT process"
 +
 +==== Activating Zend extensions ====
 +   Zend extensions are then activated **after** PHP extensions
 +''zend_extension_startup()'' is called on the Zend extensions registry
 +   Remember that Zend extensions are never sorted in any way. You must then declare them in the FIFO order in php.ini. The engine wont touch your declaration order
 +''zend_extension_startup()'' just calls for the ''startup()'' function on Zend extensions, and appends a declaration message in the Zend phpinfo() message "With module XXX" showing to end user the Zend extension is both registered, and activated.
 +   Don't be fooled here, it's not the activate() function which is triggered from Zend extensions, but the startup() function
 +
 +===== Extensions lifetime =====
 +You may know PHP's lifetime. Very basically, ''php_module_startup()'', then ''php_request_startup()'' at each request, followed by ''php_request_shutdown()'' , and then when PHP has to die : ''php_module_shutdown()''.
 +We already detailed php_module_startup() previously, to show how both PHP extensions and Zend extensions live into this stage.
 +
 +==== Request startup ====
 +Zend extensions come first, and their ''activate()'' function is called
 +PHP extensions come second, and their ''request_startup_func()'' is called, this is known as being the "RINIT" stage
 +
 +==== Request shutdown ====
 +PHP extensions come first, and their ''request_shutdown_func()'' is called, this is known as being the "RSHUTDOWN" stage
 +Zend extensions come after, and their ''deactivate()'' function is called
 +A third hook is called : ''post_deactivate_func()'', for each PHP extensions. PHP extensions are here given a chance to do some work after Zend extensions have all shut down
 +
 +==== PHP shutdown ====
 +PHP is ending, so as we are clean guys, we wrote code for PHP to clean up its environment in a right way and don't rely on the process terminating to destroy all the environment.
 +PHP extensions are first destroyed. This is done internally by relying on the Hashtable functionalities. As the ''module_registry'' containing all the PHP extensions is a Hashtable, when it is constructed for the first time, a destructor is registered. This one is called here. It is ''module_destructor()''
 +It calls ''module_shutdown_func()'' on each extension, this is called the "MSHUTDOWN" stage. Then it destroys the extension globals calling ''globals_dtor()'' and finally it unregisters the PHP functions the extension may have registered at startup.
 +Finally, if the extension were to be loaded with ''dlopen()'' (not a statically compiled extension), it is then dlunload()'ed except if the env variable ZEND_DONT_UNLOAD_MODULES is used (usefull for debugging modules with valgrind, for example)
 +
 +After PHP extensions have all been shut down and unloaded, Zend extensions will be.
 +''zend_shutdown_extensions()'' is called, and its job is just to trigger ''shutdown()'' on each Zend extension (and destroy it from the main Zend extension registry)
 +
 +===== Main schema =====
 +
 +{{:internals:extensions_lifetime.png?200|}}
internals/extensions.1365523961.txt.gz · Last modified: 2017/09/22 13:28 (external edit)