He means that in order to live life forward you must look backward so that you know how to move forward and where to move forward. Unless we understand the past, we cannot truly learn where to go in the future.
As a Christian I also believe that by looking backward I am better able to appreciate the grace that God has given to me and allow the knowledge of such grace to change how I live now and in the future. See Ephesians 2:11-22.
To answer the question posed in the title: Yes. We [ought to] live forwards but examine it backwards.