You probably can't avoid the time it takes for the pixel to physically change (~1 frame) but you can avoid all intermediate framebuffers by careful timing synchronization. E.g. when scaling, don't buffer a frame at a time, just a line.