Просмотр архива Htmlpurifier


INCLUDES, AUTOLOAD, BYTECODE CACHES and OPTIMIZATION

The
Problem
-----------

HTML Purifier contains a number of extra components
that are not used all
of the time, only if the user explicitly specifies
that we should use
them.

Some of these optional components are
optionally included (Filter,
Language, Lexer, Printer), while others are
included all the time
(Injector, URIFilter, HTMLModule, URIScheme). We
will stipulate that these
are all developer specified: it is conceivable
that certain Tokens are not
used, but this is user-dependent and should
not be trusted.

We should come up with a consistent way to handle these
things and ensure
that we get the maximum performance when there is
bytecode caches and
when there are not. Unfortunately, these two goals
seem contrary to each
other.

A peripheral issue is the performance of
ConfigSchema, which has been
shown take a large, constant amount of
initialization time, and is
intricately linked to the issue of includes
due to its pervasive use
in our plugin architecture.

Pros and
Cons
-------------

We will assume that user-based extensions will be
included by them.

Conditional includes:
  Pros:
    - User management
is simplified; only a single directive needs to be set
    - Only
necessary code is included
  Cons:
    - Doesn't play nicely with opcode
caches
    - Adds complexity to standalone version
    - Optional
configuration directives are not exposed without a little
      extra
coaxing (not implemented yet)

Include it all:
  Pros:
    - User
management is still simple
    - Plays nicely with opcode caches and
standalone version
    - All configuration directives are present
 
Cons:
    - Lots of (how much?) extra code is included
    - Classes that
inherit from external libraries will cause compile
      errors

Build
an include stub (Let's do this!):
  Pros:
    - Only necessary code is
included
    - Plays nicely with opcode caches and standalone version
   
- require (without once) can be used, see above
    - Could further extend
as a compilation to one file
  Cons:
    - Not implemented yet
    -
Requires user intervention and use of a command line script
    -
Standalone script must be chained to this
    - More complex and
compiled-language-like
    - Requires a whole new class of system-wide
configuration directives,
      as configuration objects can be reused
  
 - Determining what needs to be included can be complex (see above)
    -
No way of autodetecting dynamically instantiated classes
    - Might be
slow

Include stubs
-------------

This solution may be "just
right" for users who are heavily oriented
towards performance.
However, there are a number of picky implementation
details to work out
beforehand.

The number one concern is how to make the HTML Purifier
files "work
out of the box", while still being able to easily
get them into a form
that works with this setup. As the codebase stands
right now, it would
be necessary to strip out all of the require_once
calls. The only way
we could get rid of the require_once calls is to use
__autoload or
use the stub for all cases (which might not be a bad
idea).

    Aside
    -----
    An important thing to remember,
however, is that these require_once's
    are valuable data about what
classes a file needs. Unfortunately, there's
    no distinction between
whether or not the file is needed all the time,
    or whether or not it
is one of our "optional" files. Thus, it is
    effectively
useless.

    Deprecated
    ----------
    One of the things I'd like
to do is have the code search for any classes
    that are explicitly
mentioned in the code. If a class isn't mentioned, I
    get to assume
that it is "optional," i.e. included via introspection.
    The
choice is either to use PHP's tokenizer or use regexps; regexps would
   
be faster but a tokenizer would be more correct. If this ends up being
   
unfeasible, adding dependency comments isn't a bad idea. (This could
   
even be done automatically by search/replacing require_once, although
   
we'd have to manually inspect the results for the optional requires.)

  
 NOTE: This ends up not being necessary, as we're going to make the user
 
  figure out all the extra classes they need, and only include the core
  
 which is predetermined.

Using the autoload framework with include stubs
works nicely with
introspective classes: instead of having to have
require_once inside
the function, we can let autoload do the work; we
simply need to
new $class or accept the object straight from the caller.
Handling filters
becomes a simple matter of ticking off configuration
directives, and
if ConfigSchema spits out errors, adding the necessary
includes. We could
also use the autoload framework as a fallback, in case
the user forgets
to make the include, but doesn't really care about
performance.

    Insight
    -------
    All of this talk is merely a
natural extension of what our current
    standalone functionality does.
However, instead of having our code
    perform the includes, or
attempting to inline everything that possibly
    could be used, we boot
the issue to the user, making them include
    everything or setup the
fallback autoload handler.

Configuration
Schema
--------------------

A common deficiency for all of the
conditional include setups (including
the dynamically built include PHP
stub) is that if one of this
conditionally included files includes a
configuration directive, it
is not accessible to configdoc. A stopgap
solution for this problem is
to have it piggy-back off of the data in the
merge-library.php script
to figure out what extra files it needs to
include, but if the file also
inherits classes that don't exist, we're in
big trouble.

I think it's high time we centralized the configuration
documentation.
However, the type checking has been a great boon for the
library, and
I'd like to keep that. The compromise is to use some other
source, and
then parse it into the ConfigSchema internal format (sans all
of those
nasty documentation strings which we really don't need at
runtime) and
serialize that for future use.

The next question is that
of format. XML is very verbose, and the prospect
of setting defaults in it
gives me willies. However, this may be necessary.
Splitting up the file
into manageable chunks may alleviate this trouble,
and we may be even want
to create our own format optimized for specifying
configuration. It might
look like (based off the PHPT format, which is
nicely compact yet
unambiguous and human-readable):

Core.HiddenElements
TYPE:   
lookup
DEFAULT: array('script', 'style') // auto-converted during
processing
--ALIASES--
Core.InvisibleElements,
Core.StupidElements
--DESCRIPTION--
<p>
  Blah
blah
</p>

The first line is the directive name, the lines after
that prior to the
first --HEADER-- block are single-line values, and then
after that
the multiline values are there. No value is restricted to a
particular
format: DEFAULT could very well be multiline if that would be
easier.
This would make it insanely easy, also, to add arbitrary extra
parameters,
like:

VERSION:  3.0.0
ALLOWED:  'none', 'light', 'medium',
'heavy' // this is wrapped in array()
EXTERNAL: CSSTidy // this would be
documented somewhere else with a URL

The final loss would be that you
wouldn't know what file the directive
was used in; with some clever
regexps it should be possible to
figure out where $config->get($ns,
$d); occurs. Reflective calls to
the configuration object is mitigated by
the fact that getBatch is
used, so we can simply talk about that in the
namespace definition page.
This might be slow, but it would only happen
when we are creating
the documentation for consumption, and is
sugar.

We can put this in a schema/ directory, outside of HTML Purifier.
The serialized
data gets treated like entities.ser.

The final thing
that needs to be handled is user defined configurations.
They can be added
at runtime using ConfigSchema::registerDirectory()
which globs the
directory and grabs all of the directives to be incorporated
in. Then, the
result is saved. We may want to take advantage of the
DefinitionCache
framework, although it is not altogether certain what
configuration
directives would be used to generate our key (meta-directives!)

   
Further thoughts
    ----------------
    Our master configuration schema
will only need to be updated once
    every new version, so it's easily
versionable. User specified
    schema files are far more volatile, but
it's far too expensive
    to check the filemtimes of all the files, so a
DefinitionRev style
    mechanism works better. However, we can uniquely
identify the
    schema based on the directories they loaded, so there's
no need
    for a DefinitionId until we give them full programmatic
control.

    These variables should be directly incorporated into
ConfigSchema,
    and ConfigSchema should handle serialization. Some
refactoring will be
    necessary for the DefinitionCache classes, as they
are built with
    Config in mind. If the user changes something, the
cache file gets
    rebuilt. If the version changes, the cache file gets
rebuilt. Since
    our unit tests flush the caches before we start, and
the operation is
    pretty fast, this will not negatively impact unit
testing.

One last thing: certain configuration directives require that
files
get added. They may even be specified dynamically. It is not a good
idea
for the HTMLPurifier_Config object to be used directly for such
matters.
Instead, the userland code should explicitly perform the
includes. We may
put in something like:

REQUIRES:
HTMLPurifier_Filter_ExtractStyleBlocks

To indicate that if that class
doesn't exist, and the user is attempting
to use the directive, we should
fatally error out. The stub includes the core files,
and the user includes
everything else. Any reflective things like new
$class would be required
to tie in with the configuration.

It would work very well with rarely
used configuration options, but it
wouldn't be so good for
"core" parts that can be disabled. In such cases
the core
include file would need to be modified, and the only way
to properly do
this is use the configuration object. Once again, our
ability to create
cache keys saves the day again: we can create arbitrary
stub files for
arbitrary configurations and include those. They could
even be the single
file affairs. The only thing we'd need to include,
then, would be
HTMLPurifier_Config! Then, the configuration object would
load the
library.

    An aside...
    -----------
    One questions, however,
the wisdom of letting PHP files write other PHP
    files. It seems like a
recipe for disaster, or at least lots of headaches
    in highly secured
setups, where PHP does not have the ability to write
    to its root. In
such cases, we could use sticky bits or tell the user
    to manually
generate the file.

    The other troublesome bit is actually doing the
calculations necessary.
    For certain cases, it's simple (such as
URIScheme), but for AttrDef
    and HTMLModule the dependency trees are
very complex in relation to
    %HTML.Allowed and friends. I think that
this idea should be shelved
    and looked at a later, less insane
date.

An interesting dilemma presents itself when a configuration form
is offered
to the user. Normally, the configuration object is not
accessible without
editing PHP code; this facility changes thing. The
sensible thing to do
is stipulate that all classes required by the
directives you allow must
be included.

Unit
testing
------------

Setting up the parsing and translation into our
existing format would not
be difficult to do. It might represent a good
time for us to rethink our
tests for these facilities; as creative as they
are, they are often hacky
and require public visibility for things that
ought to be protected.
This is especially applicable for our
DefinitionCache tests.

Migration
---------

Because we are not
*adding* anything essentially new, it should be trivial
to write a script
to take our existing data and dump it into the new format.
Well, not
trivial, but fairly easy to accomplish. Primary
implementation
difficulties would probably involve formatting the file
nicely.

Backwards-compatibility
-----------------------

I expect
that the ConfigSchema methods should stick around for a little bit,
but
display E_USER_NOTICE warnings that they are deprecated. This will
require
documentation!

New stuff
---------

VERSION: Version number directive
was introduced
DEPRECATED-VERSION: If the directive was deprecated, when
was it deprecated?
DEPRECATED-USE: If the directive was deprecated, what
should the user use now?
REQUIRES: What classes does this configuration
directive require, but are
    not part of the HTML Purifier core?

   
vim: et sw=4 sts=4