viernes, 9 de enero de 2015

Node watch task problems in linux&mac

Some developer tools I'm working with lately, like node watch, listen to changes in a filesystem folder using native system APIs for this. On unix systems I had some trouble with those, but good trouble, let me explain.

In unix systems there is a limit on the number of files being watched at a time b a process. And the kernel won't allow a process to watch too many files at the same time.

This is why, we can have trouble while executing development environments based in node watch (grunt watch, gulp watch, etc) in unix (linux and mac). The name of the error has the code : ENOSPC. It mostly fails or enters in a infinite loop, for example:

grunt watch
Warning: watch ENOSPC

Solution

The solution in linux is the following:
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

viernes, 2 de enero de 2015

my nodejs tool can't use itself

So I have this nodejs application short-jsdoc for documenting JavaScript code. The strange thing is that If I want to use it in itself, this is, installing it in its own project, nodejs refuses to do so:
cd short-jsdoc
npm install short-jsdoc --save-dev
npm WARN install Refusing to install short-jsdoc as a dependency of itself
At first I was surprised and I still am because nodejs always install dependencies locally inside the project and so this situation could be managed easily, but on the other side, the user from irc.freenode.org#Node.js give me crucial tips:
1) Why would I want foo depend on foo since in foo I already have all the code of foo?
2) If you are publishing a npm module then you should expose a main .js file and you can always require that file using require('./') syntax. That will always work. Thanks !

viernes, 3 de octubre de 2014

Bash script to measure http request times


The other day I created a nice bash script for measuring web site load time. Basically you pass an url you
want to test and the number of times you want to hit it and it returns the total time and promedy.

I use curl to make the http request and it returns me the response time in seconds (like "0.123").

The hard part was to be able to do the maths in bash. First I thought I needed to parse the numbers somehow,
but there is no such thing. I found the command bc and based the script on it to do the math.


Well, here is the code:

# Usage example: 
# sh run-times.sh 40 http://wordpress.com/

times=$1

i=0
sum=0
while [  $i -lt $times ]; do

 # make the request output will be the response time in seconds. 
 output=`curl -w "@curl-format-only-total.txt" -o /dev/null -s $2`

 #multiply by 1000
 ms=`echo "$output * 1000" | bc` 

 sum=`echo "$sum + $ms" | bc`

 #round float to integer
 sum=`printf "%.0f\n" "$sum"` 

 let i=i+1 

done

mean=`echo "$sum / $times" | bc` 

echo "count: $times, total_time: $sum, mean: $mean"


domingo, 13 de julio de 2014

How to update a branch with master

I always forget how to update a branch with all my changes in the master branch. For example in github, I want to update my branch gh-pages with my latest changes in master - so the documentation and examples of my projects are updated. This is how to do it:
git checkout gh-pages
git rebase master
git push

git checkout master

miércoles, 28 de mayo de 2014

How to identify in IRC

Sometimes I need to identify myself on irc chats, mostly freenode.org for programming questions and I often not remember how to. Well this is it:
/nick cancerbero_sgx
/msg NickServ identify my_secret

sábado, 24 de mayo de 2014

Calculate average in javascript

Currently I'm doing a lot of performance improvements in one of our pages in often I need to calculate the average of the numbers in an array. For this I simply write something like the following in a javascript terminal:
(function(a){var c = 0; for(var i in a){c+=a[i]}; return c/a.length})([11.368,  10.065, 4.403])

viernes, 23 de mayo de 2014

How to measure web page load visually using window.performance API

Navigation Timing API, http://www.w3.org/TR/navigation-timing/ provides detailed  measurements about of current web page performance in terms of load time. It support server, requests and browser loading and rendering information. This standar is supported at least by chrome, firefox and IE>=9

The project https://github.com/kaaes/timing provides a visual report for seeing this information. For make it work, in your web page just add the following lines:

var profiler_js = 'https://raw.githubusercontent.com/kaaes/timing/gh-pages/profiler.js'; 
var d = document;
var h = d.getElementsByTagName('head')[0];
var s = d.createElement('script');
var t = new Date();  
s.type='text/javascript';
s.src=profiler_js+'?'+t.getTime();  
s.onload = c;
s.onreadystatechange = function(){
 if(this.readyState == 'loaded'){
  c();
 }
};  
h.appendChild(s);
function c() {
 window.__profiler = window.__profiler || function() {
  var p = new __Profiler();
  p.init();
 }  
 window.__profiler();
 __profiler.scriptLoaded = true;
}

IMHO this is the best way of measure some browser internal stuff like DNS fetching, rendering, html markup parsing, etc.

Also another way of running this profile tools but less intrusive, this is not downloading any extra file is the following:
Include the minified inline version of profile.js:
function __Profiler(){this.totalTime=0;this.barHeight=18;this.timeLabelWidth=50;this.nameLabelWidth=160;this.textSpace=this.timeLabelWidth+this.nameLabelWidth;this.spacing=1.2;this.unit=1;this.fontStyle="11.5px Arial";this.containerPadding=20;this.container=null;this.customElement=false;this.timingData=[];this.sections=[]}__Profiler.prototype.eventsOrder=["navigationStart","redirectStart","redirectStart","redirectEnd","fetchStart","domainLookupStart","domainLookupEnd","connectStart","secureConnectionStart","connectEnd","requestStart","responseStart","responseEnd","unloadEventStart","unloadEventEnd","domLoading","domInteractive","msFirstPaint","domContentLoadedEventStart","domContentLoadedEventEnd","domContentLoaded","domComplete","loadEventStart","loadEventEnd"];__Profiler.prototype.cssReset="font-size:12px;line-height:1em;z-index:99999;text-align:left;font-family:Calibri,'Lucida Grande',Arial,sans-serif;text-shadow:none;box-shadow:none;display:inline-block;color:#444;font-weight:normal;border:none;margin:0;padding:0;background:none;";__Profiler.prototype.elementCss="position:fixed;margin:0 auto;top:0;left:0;right:0;border-bottom:solid 1px #EFCEA1;box-shadow:0 2px 5px rgba(0,0,0,.1);";__Profiler.prototype.containerCss="background:#FFFDF2;background:rgba(255,253,242,.99);padding:20px;display:block;";__Profiler.prototype.headerCss="font-size:16px;font-weight:normal;margin:0 0 1em 0;width:auto";__Profiler.prototype.buttonCss="float:right;background:none;border-radius:5px;padding:3px 10px;font-size:12px;line-height:130%;width:auto;margin:-7px -10px 0 0;cursor:pointer";__Profiler.prototype.infoLinkCss="color:#1D85B8;margin:1em 0 0 0;";__Profiler.prototype._getPerfObjKeys=function(obj){var keys=Object.keys(obj);return keys.length?keys:Object.keys(Object.getPrototypeOf(obj))};__Profiler.prototype._setUnit=function(canvas){this.unit=(canvas.width-this.textSpace)/this.totalTime};__Profiler.prototype._getSections=function(){return Array.prototype.indexOf?[{name:"network",color:[224,84,63],firstEventIndex:this.eventsOrder.indexOf("navigationStart"),lastEventIndex:this.eventsOrder.indexOf("connectEnd"),startTime:0,endTime:0},{name:"server",color:[255,188,0],firstEventIndex:this.eventsOrder.indexOf("requestStart"),lastEventIndex:this.eventsOrder.indexOf("responseEnd"),startTime:0,endTime:0},{name:"browser",color:[16,173,171],firstEventIndex:this.eventsOrder.indexOf("unloadEventStart"),lastEventIndex:this.eventsOrder.indexOf("loadEventEnd"),startTime:0,endTime:0}]:[]};__Profiler.prototype._createContainer=function(){var container=document.createElement("div");var header=this._createHeader();var button=this._createCloseButton();button.onclick=function(e){button.onclick=null;container.parentNode.removeChild(container)};container.style.cssText=this.cssReset+this.containerCss;if(!this.customElement){container.style.cssText+=this.elementCss}header.appendChild(button);container.appendChild(header);return container};__Profiler.prototype._createHeader=function(){var c=document.createElement("div");var h=document.createElement("h1");var sectionStr="/ ";for(var i=0,l=this.sections.length;i<l;i++){sectionStr+='<span style="color:rgb('+this.sections[i].color.join(",")+')">'+this.sections[i].name+"</span> / "}h.innerHTML="Page Load Time Breakdown "+sectionStr;h.style.cssText=this.cssReset+this.headerCss;c.appendChild(h);return c};__Profiler.prototype._createCloseButton=function(){var b=document.createElement("button");b.innerHTML="close this box ×";b.style.cssText=this.cssReset+this.buttonCss;return b};__Profiler.prototype._createInfoLink=function(){var a=document.createElement("a");a.href="http://kaaes.github.com/timing/info.html";a.target="_blank";a.innerHTML="What does that mean?";a.style.cssText=this.cssReset+this.infoLinkCss;return a};__Profiler.prototype._createNotSupportedInfo=function(){var p=document.createElement("p");p.innerHTML="Navigation Timing API is not supported by your browser";return p};__Profiler.prototype._createChart=function(){var chartContainer=document.createElement("div");var canvas=document.createElement("canvas");canvas.width=this.container.clientWidth-this.containerPadding*2;var infoLink=this._createInfoLink();this._drawChart(canvas);chartContainer.appendChild(canvas);chartContainer.appendChild(infoLink);return chartContainer};__Profiler.prototype._prepareDraw=function(canvas,mode,eventData){var sectionData=this.sections[eventData.sectionIndex];var barOptions={color:sectionData.color,sectionTimeBounds:[sectionData.startTime,sectionData.endTime],eventTimeBounds:[eventData.time,eventData.timeEnd],label:eventData.label};return this._drawBar(mode,canvas,canvas.width,barOptions)};__Profiler.prototype._drawBar=function(mode,canvas,barWidth,options){var start;var stop;var width;var timeLabel;var metrics;var color=options.color;var sectionStart=options.sectionTimeBounds[0];var sectionStop=options.sectionTimeBounds[1];var nameLabel=options.label;var context=canvas.getContext("2d");if(mode==="block"){start=options.eventTimeBounds[0];stop=options.eventTimeBounds[1];timeLabel=start+"-"+stop}else{start=options.eventTimeBounds[0];timeLabel=start}timeLabel+="ms";metrics=context.measureText(timeLabel);if(metrics.width>this.timeLabelWidth){this.timeLabelWidth=metrics.width+10;this.textSpace=this.timeLabelWidth+this.nameLabelWidth;this._setUnit(canvas)}return function(context){if(mode==="block"){width=Math.round((stop-start)*this.unit);width=width===0?1:width}else{width=1}context.strokeStyle="rgba("+color[0]+","+color[1]+","+color[2]+",.3)";context.lineWidth=1;context.fillStyle="rgba(255,255,255,0)";context.fillRect(0,0,barWidth-this.textSpace,this.barHeight);context.fillStyle="rgba("+color[0]+","+color[1]+","+color[2]+",.05)";context.fillRect(0,0,barWidth-this.textSpace,this.barHeight);context.shadowColor="white";context.fillStyle="rgba("+color[0]+","+color[1]+","+color[2]+",.2)";context.fillRect(Math.round(this.unit*sectionStart),2,Math.round(this.unit*(sectionStop-sectionStart)),this.barHeight-4);context.fillStyle="rgb("+color[0]+","+color[1]+","+color[2]+")";context.fillRect(Math.round(this.unit*start),2,width,this.barHeight-4);context.fillText(timeLabel,barWidth-this.textSpace+10,2*this.barHeight/3);context.fillText(nameLabel,barWidth-this.textSpace+this.timeLabelWidth+15,2*this.barHeight/3)}};__Profiler.prototype._drawChart=function(canvas){var time;var eventName;var options;var skipEvents=[];var drawFns=[];var context=canvas.getContext("2d");context.font=this.fontStyle;this._setUnit(canvas);for(var i=0,l=this.eventsOrder.length;i<l;i++){var evt=this.eventsOrder[i];if(!this.timingData.hasOwnProperty(evt)){continue}var item=this.timingData[evt];var startIndex=evt.indexOf("Start");var isBlockStart=startIndex>-1;var hasBlockEnd=false;if(isBlockStart){eventName=evt.substr(0,startIndex);hasBlockEnd=this.eventsOrder.indexOf(eventName+"End")>-1}if(isBlockStart&&hasBlockEnd){item.label=eventName;item.timeEnd=this.timingData[eventName+"End"].time;drawFns.push(this._prepareDraw(canvas,"block",item));skipEvents.push(eventName+"End")}else{if(skipEvents.indexOf(evt)<0){item.label=evt;drawFns.push(this._prepareDraw(canvas,"point",item))}}}canvas.height=this.spacing*this.barHeight*drawFns.length;context.font=this.fontStyle;var step=Math.round(this.barHeight*this.spacing);drawFns.forEach(function(draw){draw.call(this,context);context.translate(0,step)},this)};__Profiler.prototype._matchEventsWithSections=function(){var data=this.timingData;var sections=this.sections;for(var i=0,len=sections.length;i<len;i++){var firstEventIndex=sections[i].firstEventIndex;var lastEventIndex=sections[i].lastEventIndex;var sectionOrder=this.eventsOrder.slice(firstEventIndex,lastEventIndex+1);var sectionEvents=sectionOrder.filter(function(el){return data.hasOwnProperty(el)});sectionEvents.sort(function(a,b){return data[a].time-data[b].time});firstEventIndex=sectionEvents[0];lastEventIndex=sectionEvents[sectionEvents.length-1];sections[i].startTime=data[firstEventIndex].time;sections[i].endTime=data[lastEventIndex].time;for(var j=0,flen=sectionEvents.length;j<flen;j++){var item=sectionEvents[j];if(data[item]){data[item].sectionIndex=i}}}};__Profiler.prototype._getData=function(){if(!window.performance){return}var data=window.performance;var timingData=data.timing;var eventNames=this._getPerfObjKeys(timingData);var events={};var startTime=timingData.navigationStart||0;var eventTime=0;var totalTime=0;for(var i=0,l=eventNames.length;i<l;i++){var evt=timingData[eventNames[i]];if(evt&&evt>0){eventTime=evt-startTime;events[eventNames[i]]={time:eventTime};if(eventTime>totalTime){totalTime=eventTime}}}this.totalTime=totalTime;return events};__Profiler.prototype._init=function(){this.timingData=this._getData();this.sections=this._getSections();this.container=this._createContainer();if(this.customElement){this.customElement.appendChild(this.container)}else{document.body.appendChild(this.container)}var content;if(this.timingData&&this.sections.length){this._matchEventsWithSections();content=this._createChart()}else{content=this._createNotSupportedInfo()}this.container.appendChild(content)};__Profiler.prototype.init=function(element,timeout){if(element instanceof HTMLElement){this.customElement=element}if(timeout&&parseInt(timeout,10)>0){var self=this;setTimeout(function(){self._init()},timeout)}else{this._init()}};


And now just run it:
 window.__profiler = window.__profiler || function() {
  var p = new __Profiler();
  p.init();
 }  
 window.__profiler();


Enjoy!