Xtream project fork. Migration to Pharo.
# Xtreams Xtream project fork of the Flexible and scalable streaming for Smalltalk, ported for Pharo >=6.0 . # Synopsis Xtreams is a generalized stream/iterator framework providing simple, unified API for reading from different kinds of sources and writing into different kinds of destinations (Collections, Sockets, Files, Pipes, etc). Streams themselves can be sources or destinations as well. This allows to stack streams on top of each other. At the bottom of such stack is some kind of non-stream (e.g. a collection), we will call it a terminal. Directly above that is a specialized stream providing a streaming facade over the terminal. The rest of the streams in the stack we'll call transforms. Their primary purpose is to perform some kind of transformation on the elements that are passing through. Application code interacts with the top stream of the stack the same way it would with any other stream (or stream stack) producing/consuming the same elements. The goal of the framework is to provide consistent behavior between different stacks so that the application can treat them the same way regardless of what exactly is the ultimate source or destination. For example if the application code analyzes binary data, it should be able to treat the source stream the same way if it is a simple stream over a ByteArray or if it is a stack that provides contents of a specific binary part of a mulitpart, gziped, chunked HTTP response from a socket. Xtreams is an attempt to achieve this goal in a scalable and efficient manner. Xtreams are currently maintained in Cincom Smalltalk Public Repository. Primary development is done in VisualWorks but there are several ports in varying state of completeness available. There is a port to Squeak/Pharo underway (thanks to Nicolas Cellier) available on squeaksource.com. The same port with some tweaks (available in the same repository) was successfully imported into Gemstone/S (courtesy of Dale Henrichs). A port to Smalltalk/X is underway as well (thanks to Jan Vrany). See also https://github.com/mkobetic/Xtreams. The project consists of several packages: | Package Xtreams | Notes | |----------------------|------------------------------------------------------------------------------------------| | Core | Defines the API and core classes | | Terminals | Streams for all supported terminals (Collections, Blocks, Files, Sockets, Pipes, etc) | | Transforms | Core transform streams (collection style, encodings, marshaling, etc) | | Substreams | Streams embedded in other streams, slicing and stitching streams | | Multiplexing | A substreaming protocol for sharing a paired read/write stream | | Crypto | Cryptographic transforms (hashing, encryption, etc) | | Compression | Compression streams (e.g. deflate) | | Xtras | Additional non-core transforms (e.g. streaming over external heap) | | Parsing | PEG parsing Includes parsers/generators (PEG, Smalltalk, Javascript, Wiki, etc) | The packages are bundled together with associated Test packages in a bundle called Xtreams. This bundle also includes some example applications, e.g. IRC client. Other applications include Polycephaly2, a flexible SSH2 client/server, or the new SSL/TLS implementation for VisualWorks. There is not (yet another) project specific forum for people to subscribe to. Unless the xtreams traffic picks up to the point that having a dedicated forum would be useful, let's make use of the existing smalltalk ones. c.l.s and vwnc are the ones where you're most likely to reach Martin or Michael. # Installing Xtream Download a Pharo ``` Metacello new baseline: 'Xtreams'; repository: 'github://sbragagnolo/Xtreams'; load. ``` # Basics - Xtream Core Xtreams is a generalized stream/iterator framework providing simple, unified API for reading from various kinds of sources and writing into various kinds of destinations (Collections, Sockets, Files, Pipes, etc). Streams themselves can be sources or destinations as well, allowing to stack streams on top of each other. At the bottom of such stack is some kind of non-stream (e.g. a collection), we will call it a terminal. Directly above it is a specialized stream providing a streaming facade over the terminal. The rest of the streams in the stack we'll call transforms. Their primary purpose is to perform some kind of transformation on the elements that are passing through. Application code interacts with the top stream of the stack the same way it would with any other stream (or stream stack) producing/consuming the same elements. The goal of the framework is to provide consistent behavior of different stacks so that the application code can treat them the same way regardless of what exactly is the ultimate source or destination. For example an application analyzing binary data should be able to treat the source stream the same way if it is a simple stream over a ByteArray or if it is a stack providing contents of a specific binary part of a mulitpart, gziped and chunked HTTP response coming through a socket. Xtreams is an attempt to achieve this goal in a scalable and efficient manner. Note that the sequence of streams in a stack cannot be completely arbitrary. Each transform constrains the type of input it accepts and the type of output it produces. Since the adjacent streams in a stack are linked so that input of one is connected to the output of the other, they need to produce/consume same type of elements . For example, a "character encoding" read transform converting bytes to characters can only sit on top of a stream that produces bytes (0..255). These constraints are not checked when the stack is being constructed, it is up to the application code to make sure the streams in the stack are compatible with their adjacent streams. The framework is split into several packages: | Package Xtreams | Notes | |----------------------|------------------------------------------------------------------------------------------| | Core | Defines the API and core classes | | Terminals | Streams for all supported terminals (Collections, Blocks, Files, Sockets, Pipes, etc) | | Transforms | Core transform streams (collection style, encodings, marshaling, etc) | | Substreams | Streams embedded in other streams, slicing and stitching streams | | Multiplexing | A substreaming protocol for sharing a paired read/write stream | | Crypto | Cryptographic transforms (hashing, encryption, etc) | | Compression | Compression streams (e.g. deflate) | | Xtras | Additional non-core transforms (e.g. streaming over external heap) | | Parsing | PEG parsing Includes parsers/generators (PEG, Smalltalk, Javascript, Wiki, etc) | ## Stream Creation Xtreams strictly separate read and write streams. This allows for simpler API with better consistency. If necessary it would be quite simple to create a wrapper that holds a separate read and write stream and dispatches the read operations to one and write operations to the other. Streams are primarily created by messages with -ing suffix on the first keyword (it often is a simple unary message). A read stream on top of an arbitrary source terminal is created by sending message #reading to it. The source can be a collection, an IOAccessor (socket, pipe), aFilename, etc. For an exhaustive list of supported types of sources see package Xtreams-Terminals. If #reading is sent to a read stream, the same stream is returned. Similarly, a write stream is created by sending #writing to a destination terminal. Generally the same kinds of objects can serve as sources or destinations, Xtreams-Terminals provides further details on what those are. Message #writing sent to a write stream returns the same stream. ## Basic Operations Here are few simple examples showing equivalent code in classic streams and xtreams: "classic" ``` 'hello world' readStream next: 5. ``` "xtreams" ``` 'hello world' reading read: 5. ``` "classic" ``` String new writeStream nextPutAll: 'hello'; contents ``` "xtreams" ``` String new writing write: 'hello'; close; terminal ``` Nothing terribly surprising here, just some API changes. Note that xtreams do not provide the #contents message. It's a read operation and as such it doesn't really fit a write stream. Consequently it only works in specific circumstances (e.g. above), but not in a case like this for example: ``` (ByteArray new withEncoding: #ascii) writeStream nextPutAll: 'Hello'; contents ``` Instead, in Xtreams, if you close a write stream on a collection it will trim the underlying collection down to the actual content. So the collection itself corresponds to stream #contents after closing. This provides equivalent capability within the constraints of generally applicable write stream API. Message #terminal can be used to retrieve the underlying terminal from a stream or stream stack. Message #conclusion can be used as a convenient shortcut combining #close and #terminal together. Here's a rough correspondence table of the most common API | classic | xtreams | |-----------------