This is just a shower thought I had the other day. As you will see, I know nothing about Physics, Astronomy etc, so please don't be too harsh.
Here goes:
Intuitively, it seems to me the observable universe can be sliced into a 3d grid with coordinates and we can track the relative position of things within it over time.
Since time and space are intertwined, if we had some means to travel back to a time-position point in the past on that grid, we'd accomplish time travel. Yay.
But... we don't know where we are in relation to the rest of the universe.
Even if our galaxy or cluster or whatever was moving in a given direction within the context of the observable universe, within the rest of the universe, we could still be moving net the opposite direction depending on what direction our observable universe is moving within the rest of the universe.
And since the rest of the universe isn't observable, we could never hope to construct any 3d grid with absolute time-position coordinates of anything.
Isn't this one (of many) reasons why time travel would be impossible?
This allows you to work in a relative time-position instead of absolute time-position.
That is why you can't go in-out of fort Knox Vault stealthily with a time-machine without at some point in time having to go through the door.
But I've always thought a similar thing, that in Sci-Fi shows and so on, jumping in time would mean you would have to move in space too!
If you jumped back in an hour you'd be floating in space...
To time jump in the same building would require all sorts of prior knowledge about the 3D space... even the ground could be higher or lower... let alone the position of the actual planet.
Sci-Fi treats time and space as different things!
Unfortunately when time is taken into consideration, we're looking at a 4D (at least, but let's be minimalist) space-time.
It seems generally accepted that we are living in a multiverse, so that even if time-travel was possible, even the act of breathing would enforce a move to a different universe from the universe where you had started your time-travel, and which you could never return to.
So, you can go back and kill Hitler (or even your own grandmother). But you can never go back and say "See what I did!".
According to your hypothesis, if you build a time-travel-space-ship you could time-travel to other points of that universe, being your only limitation keeping yourself inside of that universe.
But you are not thinking with portals yet.
We "know", thanks to String Theory, that there are other spacetime dimensions "curled up" around the usual dimensions that we experience. The Large Hadron Collider has been running experiments to try to prove that by provoking collisions between particles and see if some of the energy gets "trapped" in those extra dimensions.
If the theory is true, that would mean that we are constantly time-traveling back and forth through time and space on multiple directions... just not the ones that we experience. From that dimensions point of view we might look like sliced beings that appear and disappear on many different points at the same time and in many different times at the same point.
If our time-travel machine manages to get us curled in on those dimensions we could time-travel in the usual dimensions by moving inside the curled ones following a different time (assuming that this other time dimension moves backwards with respect to our time).
From the outside it would look like our particles are rotating and becoming strings onto themselves like thread balls.
From the subject's point of view, the universe would bend around him, become dark and cold. And then, everything would disappear. His "experience" would probably be closer to "non existence". It would be the time machine what would have had to have everything ready to start and finish the trip on the right direction and time length.
Time travel will only be possible to points where your own self was in the past and only distanced by short time lengths.
Mind that every particle would travel inside the curled dimensions attached to it, but our bodies are built by many substances and structures that our own body creates from food, water and air. Each atom on our body would time-travel to where it was at the time you travel.
It might seem like a very limiting factor but this is because we are only thinking about the first human-like time-travelers. However, machines could time travel without almost any problem to the exact point where they were assembled.
The main problem would be to send new information since any new structure would disappear (drawings couldn't go back any further than where they were drawn and bits in hard drives would go back similarly to their previous statuses). But I'm sure we could NOT send complete structures as a way to send new info.
For example, we could remove parts of a radioactive sheet of metal (imagine it just by scratching the surface and removing enough atoms). We could send only the sheet of metal and not the scratched parts. Back in time, we could observe that same sheet of metal and it won't be scratched at that time... but the atoms that came from the future would have experienced more radioactive decay since they existed for a longer time.
Is this a paradox? why was that sheet of metal not present that same radioactive decay before we scratched the surface?
I don't know. But now you are thinking with portals.