Note: This is the third (and last) part of a tutorial, read first part and second one in case you haven’t.
What didn’t make it
Before focusing on minimizing the size of the finished 1k notebook design, I made several experiments on different methods of texture generation to try to simulate the paper. Unfortunately neither of them was enough small and give good visual results at the same time. So I decided to explain it here, next one was the best style looking of all the ones I tried.
The texture generation
The algorithm is based on a different step sequence applied in order. First step is to generate a bunch of black to white random gradient covering the whole canvas. Each one is copied to a second canvas with a ‘difference’ brush applied. This means, before pasting the gradient to the canvas, check the destination and origin pixels and compare them, store the absolute difference between the two on the destination canvas. The code to do this is really small.
var Q=a.createImageData(W,H);
var O=Q.data;
for (var j=9;j--;) {
var G=a.createLinearGradient(
Math.random()*W,
Math.random()*H,
Math.random()*W,
Math.random()*H);
G.addColorStop(0,'#FFF');
G.addColorStop(1,'#000');
a.fillStyle=G;
a.fillRect(0,0,W,H);
I=a.getImageData(0,0,W,H).data;
for(i=n;i--;)
O[i]=(i+1)%4?Math.abs(O[i]-I[i]):255;
}
The single gradient generated on every iteration is like this one.
By repeating this with the difference brush, we got the texture step on the left. The next step is to apply several emboss filters in sequence (2 in this case). After this, the paper texture is generated. The result is on the right.
Gradient Num:
The emboss filter is applied by using the 2 canvas and the emboss matrix filter, with different weight value on each case.
[0 0 0]
[0 1 0]
[0 0 -0.5] // Emboss filter matrix
// p & q are Imagedata.data
// r is the weight value
j=W*4+16;
for (i=0; i<n; i+=4,j+=4) {
q[i]=q[i+1]=q[i+2]=r-(p[i]-p[j]*.5);
q[i+3]=255
}
Once the paper is generated, the blue and red lines are drawn using alpha blending. The top holes are made by drawing black circles and the texture is complete.
Unfortunately, the whole code size for this generator is too much to fit together with the font data and rendering algorithm. That’s why was not included in the submitted version to the contest.
Note: This is the second part of a tutorial, read first part one in case you haven’t.
Compressing the data set
The delta coordinates stored in 16-bits Unicode characters method was not efficient if the prod is going to be crushed by a semantic compressor like JSCrush. By using this kind of characters, the compressor will not be able to collect repeating ones together with the used in the code itself. I decided to pack the data into simple chars, 8-bits. And then headaches and problems started. Looks like javascript doesn’t have a full compatible ASCII 8-bits table, and when using some of them (specially over 127), they got encoded into multi-byte 16-bits one, increasing the size of the prod.
While messing with this issue, I noticed a particular thing over the coordinates set. Check the following data plot, they’re all the coordinates scattered in a X/Y plane.
The majority of them are in the [-3, 4] range for the X axis and [-7, 8] range for the Y axis. There are only 8 deltas out of those ranges. To store the X coords, only 3 bits are needed [0, 7], or [-3, 4] subtracting 3 to the limits. For the Y coords, 4 bits are needed [0, 15], or [-7, 8] applying the same rule. 7 bits in total, storing all of them in simple 7-bit chars was achievable.
At this point, I was really tired of manipulating values manually in the array and checking the results. Also lot of deltas were not needed, Hershey font over use lot of points to have a good glyph rendering, specially for big font sizes. But in this case I knew the font size was not going to be really big, so I was able to discard many of the intermediate ones specified. With all this manual tweaking, it was clear I need a font editor. I made it fast, using javascript.
With the editor in place, the data manipulation went very fast. Tweaking out of range coordinates and changing the starting point of several glyphs to fix the kerning problems were easy tasks. The dataset was already inside the 7-bit storing ranges.
Still around 500 coords, equal 500 chars for the whole font set. 100 bytes saving achievement. Looking into the glyph shapes, is easy to notice lot of the glyphs segments are repeated between several of them.
For example, the chars ‘f’, ‘h’, ‘k’ and ‘l’ shares the same upper part, the same happens with the bottom of ‘f’, ‘g’, ‘j’, ‘y’ and ‘z’. When these segments are stored only once, the code can draw them from callings on different places of the dataset. The only thing I need is a way of storing this “segment calls” inside the data. If you look into the plot before, there is a big empty area with no coords usage below the -3 value in the X axis. This means the glyphs are not using any of the [-3,-1] to [-3, -7] points. I was able to use this points to store up to 7 different segment calls, the drawing code will draw the complete segment when finding any of this point in the drawing loop.
I tweaked the editor with this improvement very fast. Several glyphs shapes were changed to take usage of the segment defined. Letters ‘r’ and ‘q’ made a radical change of shape. During this update, many points were removed, as the font rendering size was already small and they were non increasing the drawing quality. The rendering loop was jumping to the next glyph starting point after drawing the current one, there was an array variable storing all the glyph widths. The font editor allowed me to tweak all glyphs so they can start and finish on the same Y height always, in this way the rendering loop will be finishing all glyph drawing already with the correct width. The problem was the ‘x’ character, I had to change it completely to a continuous drawing one to fit into this modification. With all these improvement in place, I achieved to reach 246 chars only. Check the rendering difference between the original Hershey font and the compressed one:
That’s the complete dataset, 246 (there are characters not appearing because they’re on the lower part of the ASCII table, escape ones and so on).
The segments appears in different color. First one is the red one, used 7 times. Second is the green, 4 times. Third the black, five. Next the orange one, six times. Gray one 7 times. Yellow and violet are the ones least used, only 3 times. All the blue lines are the data not belonging to segments, needed to complete the glyphs.
Scripting animation
Once the compression was finished, I got enough space available to program the code for the rendering and the notebook styles. The code to do the font rendering is very simple, all the chars are translated to the delta coords needed in a sequence array (Z), considering the segment calls. Then a loop is getting values from the array and drawing using the canvas lineTo method until the array is empty.
There are several special cases controlled in the code, like pushing keys out of the [A..Z] range or the dots for ‘i’ and ‘j’ glyphs.
The flower glyph
During all the developing time, I was in touch with my good friend Javier Guerrero (aka Infern0). He was testing and checking for bugs, and at the same time giving me feedback on improvements or better look styles. To fit into the contest topic “spring”, a hand-written flower can’t be better election. I sent to Javi all segments used in the compression stage, he came back to me very fast with a proposal.
Just lot of ‘c’ segments and the lower part segment from the ‘j’. Special cases were added to the rendering loop to move the drawing coordinates and make the correct jumps for drawing the segments in the flower shape.
That is the complete explanation of the released prod. In the next chapter, what didn’t make it. A nice paper texture code generated which couldn’t fit in the 1024 bytes size.
Everything started just after JS1k 2012, “Love”. I was making some experiments with the possibility to implement a full parametric vector-font in canvas. The first approach was a very basic font face, made with straight vectors. Write in the input field to update the font rendering.
The encoding of this font was only 161 chars in total, including the string declaration with the split(0) to make it as array of strings.
Every letter is a set of lines drawn in sequence. The start and end points of every line are encoded using the next two dimensional grid definition, common for every glyph:
The code to draw the characters from the data is very simple. Loop through all the chars encoded, drawing lines from point to point decoded. There are four special cases, for chars i,j,t and f. The code draws an additional line for those ones to finish the drawing.
var P=f[t.charCodeAt(i)-65];
if (P) {
for (j=1;j<P.length;j++)
drawline(P.charCodeAt(j-1)-65,P.charCodeAt(j)-65);
if (t[i]==='I'||t[i]==='J') drawline(3,4);
if (t[i]==='F'||t[i]==='T') drawline(3,5);
}
With this experiment was clear the method got possibilities. Total code size was much lower than 1024, so why not try it with a more complex font face?
The Hershey font
Once JS1k 2013 was announced, I came back to the experiments and tried to make something to submit. More or less at the same time, Jari Komppa released the demo Litterae Finis, where some scripting fonts appeared in text mode. He made a breakdown, explaining the secret fonts used, the Hershey fonts.
The font collection was developed in 1967 by Dr. A. V. Hershey. The original data format is really cryptic, fortunately Jari developed a decoder and made it available to the public. The script face was perfect for the task, a handwriting font with a single vector trace.
The Hershey script font data for the character “b” is an array of coords. Every pair of numbers is a (x,y) couple. The vector lines are defined by two consecutive couples, this helps a lot to draw fonts with non continuous traces. But in my case the target is to have a continuous single trace, so a very fast compression can be achieved by removing this:
c67 = [0,20,2,17,2,17,5,12,5,12,6,10,6,10,7,7,7,7,7,5,7,5,6,4,6,4,4,5,4,5,3,7,3,7,2,11,2,11,1,18,1,18,1,24,1,24,2,25,2,25,3,25,3,25,5,24,5,24,7,22,7,22,8,19,8,19,8,16,8,16,9,20,9,20,10,21,10,21,12,21,12,21,14,20]; // before
c67 = [0,20,2,17,5,12,6,10,7,7,7,5,6,4,4,5,3,7,2,11,1,18,1,24,2,25,3,25,5,24,7,22,8,19,8,16,9,20,10,21,12,21,14,20]; // after
Next steps to continue compressing the data are to split the array in two (x and y coordinates separated) and apply a delta difference between consecutive coordinates. The drawing algorithm will make a cumulative sum to generate the next coordinate.
The first coordinate of every glyph is always a jump to the drawing starting point. This is controlled by the drawing algorithm with an special case coded for the first coordinates.
Try clicking at the nodes in the previous canvas. The details on how the “b” char is encoded are shown. We got a data set with all (x,y) values in a 4 bits range (-8 to 7). Before encoding the coordinates, they’re converted to only positive ranges by adding 8. In this way they fit perfectly in 4 bits (0 to 15). The two couples of every vector are stored in an Unicode char (16 bits). After doing this for the whole 26 glyph set (from A to Z), we got:
300 unicode chars, this means 600 bytes. Still too big, it was clear further compression techniques were needed. In the following animated drawing of every glyph, details as the initial kerning jump and single-trace style of the font are shown.
Check the jump to the right made at the beginning by characters “a”, “c” or “g”. This is also happening with “f”, “j” and “p”, braking the continuous kerning of the font. In these cases, glyph coordinates were modified by hand to fix it.
How to reach the 1024 bytes target with further compression techniques and many more things implemented in this demo, in the next chapter. Thanks for reading!
This is a collection of different methods, tweaks or tricks to reduce the size of any javascript source. They’ve been collected here from different online sources, mostly the entries to JS1k contest and different development blog posts.
These are not “good programming” habits but totally opposite. A single place where to find golfing techniques detailed, only that. Check it here.
JS1k runs again this year, I’ve developed The 1k Notebook, using JSCrush to compress the source and reach the 1024 chars target. The compressor works by searching repetitive patterns in the source and replace them by tokens of chars in a single string. Then, one short piece of code is added at the end to uncompress the string and execute it, more detailed explanation here. This technique achieves high compression rates when the demo uses lot of long function calls or many similar code patterns.
Got lot of feedback asking for the source code of my demo, so a blog post explaining how it’s made is coming (all questions will be answered there). What annoys me are the comments about “lost of pure scripting” and “the contest lost the magic when encryption was allowed”. WTF?
Introducing JSUncrush. It takes any compressed source and output the original script. Un-minify by hand to learn!
After years of no coding any chunk of code, last year I decided make something for js1k contest. Javascript is not a programming language I love so much for size coding competition like this. I prefer compiled coding, where you can optimize for sizing using op-codes and data types more than semantic-based tricks like the case. Beside that, it’s funny and addictive! Nice language tricks can be implemented, i’ve spent more than one day renaming, un-looping and re-sorting code to win some bytes.
This time contest had the “Love” theme, not a mandatory requirement in the rules, but good if the production has any kind of relationship with the theme. Texel, yesteryear friend and same city neighbor, already released his Rose when my coding started. My prod needed to be classy and original to have any chance.
I took the decision of doing something with particles. Particles are always impressive when there are complicate interactions between them. Easy to code with minimal interaction, increasing difficulty when introducing physics like collisions, springs, temperature, densities or viscosity effects.
The code is a very basic parameters init plus a single loop function.
123 chars to compress several Math object calls to single letters, adjust the canvas size and change the composite render type to “lighter”. This last operation will make possible the blending between particles.
All particles data is stored in a single float array (p), with 9 positions for every particle. First three ones for X,Y,Z coordinates in space, next three for velocities and the last three for the next calculated movement (delta). By looping through all the array on every frame we can achieve the main particle animation in four steps:
Checking if the particle is within the render limits, made with 6 basic comparisons. If the particle achieves any bound limit, the position is fixed to this limit (kind of crash with the walls).
Calculate if the particle touchs the “heart” 3D equation:
u=1.5; x*=x*u; y*=u; z*=z*u;
if (m.pow(x+9*z/4+y*y-1,3)+x*y*y*y+9*z*y*y*y/80)>0)
Interact with all the other particles in the array. By looping again through the remaining positions in the array, and calculating the distance between current particle and the one checked, we can use a tweaked Hooke law to achieve particle interaction and simulate kind of liquid or flow process. The result is stored in the lasts 3 positions of the array reserved for the current particle, this is the delta movement that will be processed on the next frame.
J=p[T++]-p[I++]; K=p[T++]-p[I++]; L=p[T++]-p[I++];
D=J*J+K*K+L*L;
if (D<0.01&&D) {D=m.sqrt(D); M=(.3-D)*.01/D;}
Make the render. Update the array (positions and speeds), make the 3D rotation in space and project the 3D coordinates to the screen with the focal distance to the camera. Once we got the 2D coordinates the particle is drawn as a circle with a gaussian gradient. This gradient combined with the globalAlpha property set before, makes a blending between particles when they’re close together, simulating a fake liquid.
And that’s it. I’ve used JSCrush to compress the code, getting it down to 1024 bytes.
Hope you’ve enjoyed the explanation. This year the contest runs again with “spring” theme. I’ve already submitted a new demo, completely different from anything done before, check it!
New version of the javascript coding challenge JS1k started. I’m in with my prod Liquid Heart. Deadline gone, waiting for the winners now. My favorites:
Montecarlo sampling of surface equations tuned until reaching the rose shapes shown. Check also previous static version here and how is made on his blog also.
Really great particle simulation with repelling and attractors fields. Speed optimized for high particle quantity. Do not forget to check full power experiment.
Hybrid and Electric vehicles, by design, introduce many new electronic components into the bill of materials and account for up to 30% of the vehicles total cost. Years ago, Toyota was the only manufacturer with a hybrid car in mass production, but Toyota’s policy of integrating the supply chain into their own structure and utilizing only local factories made it impossible to develop sub-suppliers specializing in these new components. In 2011, all car manufacturers have one or several projects based on HEV technology. This requires the development of a complete supply chain focused on these new key elements for electric power.
Toyota Prius
Batteries, from the beginning, received the majority of attention and focus since they represent the single largest cost in these new power trains. Alliances and joint ventures have been created between car manufacturers and standard battery manufacturers with targets of lowering costs, reducing weights and improving energy output. All battery manufacturers are looking closely at the innovations, patent applications and movements of their competitors. The companies who are quick to market with any advantages or improvements in battery technology and performance will see greater profits – a very hard challenge for an industry that has suffered from a lack of innovation for more than 20 years.
A full electric car has a bill of materials with an average of one third of the number of components than a gasoline or diesel powered automobile. The number of components required for the power train, by itself, is reduced to the battery, electric motor, charger and power converter. The main costs and margins are now focused on the battery and the electric motor set rather than in the internal combustion engine. Automotive companies are getting involved in developing these key elements on their own as a key strategy to control and keep in house these larger value-added components.
Plug-In Hybrid Power Train Design
Tier 1 suppliers are looking for other elements in the powertrain to realize profits and these include the power converters and battery chargers. These elements transfer the energy in the powertrain from the battery to the motor, charge the battery and generate the low voltage power network to feed the car’s electronic systems (lights, entertainment system, heating, etc.). The basic technology used in these power converters is very well known as, in large part, the converters are switched mode power supplies with small form factors and high efficiencies. High efficiency power supplies have been developed for many years for use in other markets including UPS systems, solar panel generators, wind power energy and industrial motor drives. Companies producing converters for these applications, including Eaton and Bosch, can become new Tier 1 suppliers for these HEV platforms. Their challenges will be in having to design and manufacture the converters for onboard installation and to be qualified for use in automobiles.
In the same way, electronic Tier 1 suppliers for gas and diesel powered automobiles are already transforming themselves to develop and manufacture power converters and battery chargers. Lear, Valeo, Delphi and Continental are excellent examples of these Tier 1 suppliers and each now have converters and battery chargers in their product portfolios and many will be in new HEV cars to be launched in the coming years.
Delphi DC/DC Converter
Key electronic components used in these new HEV power assemblies are commonly utilized in the industrial markets and include high power switching semiconductors, high energy capacitors and high power transformers and inductors. In the automotive marketplace, electrical and mechanical constraints are dictated by the size and volume available for the assembly, cost targets and required efficiency ratings. Component manufacturers (Tier 2) have to adapt their business models to meet the demands of this new potential market. The heart of the component technology is already well known and in use in industrial applications but the new requirements of HEV make it necessary for a complete reinventing of these components.
A high power planar transformer from PREMO Group
Infineon and Semikron are two power management IC manufacturers who have focused on the HEV market for the last four plus years. Both have launched automotive qualified semiconductors ready for integration into HEV converters and have dedicated departments to support car manufacturers and to develop R&D kits and application notes for their MOSFETS/IGBT’s. Their target is clear: to make completely off-the-shelf parts available and to reduce the overall costs of their automotive customers.
In addition to the active components used in the converter, there are many required passive components that have been the focus of Tier 2 suppliers. One such classification is inductive components including power transformers, power chokes, current sensors, EMI filters and auxiliary signal transformers. These types of components are commonly used in industrial applications but have never been before installed in automobiles. PREMO is a Tier 2 inductive component manufacturer that has experienced a major transformation in their business model due to this evolution. Four years ago, Premo anticipated the future needs of the automobile market for these types of components. Over this four year period, Premo’s know-how, expertise and inductive components portfolio for industrial applications has led them to be able to deliver standard, agency-qualified solutions for automotive customers.
PFC Choke Developed for Automotive Applications
Onboard battery chargers for plug-in vehicles introduce a new technology challenge for these companies. The charger connects the automobile to the domestic power network, merging the requirements of the industrial world with the automotive world. The charger must comply with all EMC requirements from these industrial standards while, at the same time, fulfilling the lifespan, environmental, robustness and safety requirements of the automotive market. PREMO is the first Tier 2 component manufacturer to make available to the HEV market a complete off-the-shelf EMC Filter for battery chargers with complete AECQ-200 qualification. This filter is delivered in a single aluminum box with connectors, simplifying the assembly operation for the charger manufacturer.
PREMO EMC Filter for Battery Chargers
Opportunities are in the air. Even with the conservative numbers the automobile manufacturers are predicting for HEV unit sales in future years, the component suppliers will need to initiate a complete conversion and re-tooling of their factories to reach the capacity and output that will be required to support these programs. The production of several hundred thousand EMC Filters of the type required for battery chargers is not a simple task and no manufacturer has ever built one previously with the productivity and automation level required by the automotive market. This is a very big challenge for component companies who may be used to having a high operator workforce for their industrial customers.
Esta entrada se publicó en Junio de 2005 en Necoblog:
Bueno, ya llevo aquí 3 semanas y me ha pasado de todo. La verdad es que China es un país muy grande, aquí todo está muy lejos. Trabajo en Wuxi, una ciudad de cuatro millones y medio de habitantes cerca de Shanghai. Cerca significa 2 horas en coche si pillas bien la autovía.
La contaminación aquí es brutal. El desarrollo que están llevando a cabo los chinos en infraestructuras e industria lleva una velocidad vertiginosa. Esto hace que no tengan cuidado a la hora de quitar la mierda. Aquí todos los ríos y lagos son marrones, las desembocaduras siguen marrones hasta bien entrado el mar y el cielo siempre está con esa neblina que no deja ver el sol. Cuando vas en coche y te aproximas a una ciudad grande, la nube marrón que flota sobre ella es apreciable a simple vista. Si llueve la lluvia es mierda literal, no barro ni agua sucia, es mierda.
La contrapartida es que tienen una industria de fabricación que crece y crece sin parar. Vas por la autovía y los anuncios en las vallas publicitarias son de rodamientos, cilindros, tornillos, etc. En la estación de trenes de Shanghai, al bajar del vagón no hay anuncios de colonias, ropa o coches. Hay anuncios de máquina herramienta, de proveedores de acero y de fresadoras.
El tráfico en Wuxi y en China en general es totalmente caótico. Mi carné de conducir español no sirve aquí, pero aun si sirviera no podría conducir. Tendría un accidente el primer día. Los semáforos están en el otro lado del cruce, la preferencia no sigue la normativa de las señales o los semáforos. La norma es la siguiente: camión gigante, camión pequeño, taxi pijo, taxi Santana 2000, coche, moto, bici, chino. En ese orden. Si un camión entra en el cruce y un taxi está cruzando, pito al canto y el taxi se para. Si un chino cruza un paso de cebra y viene un Santana con un cliente, a joderse tocan y la pelas en medio de la carretera si tiene ocho carriles. Fijaos que al extranjero ni lo he nombrado, así que os podéis imaginar lo que hay que liar para cruzar una avenida de las que hay por aquí.
Para desplazarnos en el tiempo libre usamos los taxis, normalmente Volkswagen Santana 2000, que son los que más abundan. Si quieres ir a un sitio normalmente lo dices y te entienden porque no suelen ser sitios raros. A menos que te toque un chino incompetente o que no sabe leer, entonces mejor bajarse del taxi y pillar otro. Anoche nos tocó uno de esos. Resulta que queríamos ir al centro a comernos una pizza y tomamos un taxi. Nada más subir ya vimos que el tío era un inútil porque le dijimos:
— “Chélaton!”
Que es uno de los hoteles famosos del centro (Hotel Sheraton, lo de la erre no es ninguna leyenda urbana, aquí el tren es “tlen” y cabrón es “cablón”) y ni puta idea. Total, que sacamos el papelito y tampoco se ve que sabía leer un pijo. Al final le dijimos otro hotel que está en la misma calle y eso si que lo entendió. Pero mi intención era que nos dejara en el Sheraton al llegar a la calle, así que cuando estábamos en el semáforo vi el Sheraton a la derecha y el otro donde íbamos a la izquierda. Al momento saqué el brazo por la ventana y me puse a pegarle voces:
— “Chélaton!! Chélaton! Go! There! Allí!!!”
Y el chino venga a decir chineces que nos íbamos para el otro hotel. Aquí los semáforos en vez de un circulo grande verde o rojo tienen el tiempo, como los marcadores de posesión del baloncesto pero de colores. Quedaban ya un par de segundos para que arrancara y yo veía que el chino seguía con la intención de salir hacía el sitio contrario.
— “Allí cabrónnnn!!!” con el brazo cada vez más fuera del taxi… “Tira pa allaaaaaaa”
— “Chélatonnnnnnnnnnnnnnnn!!!”
Cuando se pone verde, el joputa el chino sale pa donde le da la gana. Al meter el brazo dentro, como ya llevaba inercia, le arremeto en el hombro bien al chino, pa que se entere bien, a la vez que le digo:
— “Pujao!, Pujao!, hijoputa, TIN TIN TIN TIN!!!”
Pu hao es muy malo en chino y tin tin es para el coche. Le metí un buen meco y el cabrón empezó a reirse, tin tin tin cabrón, para que nos bajamos que nos mandas a tomar por culo. Al final se paró le pedí la puta fapiao (factura) y nos fuimos al Chélaton de los cojones andando.
Hola de nuevo. Esto se puede llamar la vuelta de Necoblog, o la vuelta de Neco a esto de escribir. Aquí pondré posts del antiguo blog, quizás retocados o extendidos, según me de. Además de los nuevos que vayan surgiendo. Los que estéis buscando el dominio antiguo, olvidadlo, está squateado. Ahora apareceré por este, que quienes me conozcan entenderán bien. Un saludo y feliz Navidad.
