Is this the memory usage figure?
SQL will allocate memory as it needs it, over some days it will increase (up to the maximum limit that is configured). Once it reaches maximum it will not reduce again, so it is normal to see that SQL is using very little memory just after a restart.