Let's break it down...
A complex layout may look great on a desktop monitor or laptop, but when viewed on a phone, especially in portrait mode, it may look cramped and jumbled. A phone in landscape mode may fare better, but then there isn't much vertical space, and users will be scrolling constantly. The best thing to do is to simplify your layout for mobile devices.
A site's navigation, for example, could either go across the screen or be a stack of link buttons when viewed on a phone. But when you move to a larger display, you can rearrange the layout. This lets you take advantage of the wider space and make the layout more interesting.
Desktop Example
Smartphone Example
Remember, moving entire sections of your page could cause the orientation of certain style choices to change as well. For example, if you have a div with borders above and below it and move it off to the left for wider screens, that border should be changed to appear on the div's right side.
Question
What is the best way to ensure the best layout and usability when designing a responsive site?
The best way to ensure that your work displays properly is to test it as you work! All web browsers have inspection tools that allow you to see the viewable browser widths and code as you resize it. There are also several free testing platforms that allow you to see your work on multiple devices to ensure compatibility. Bottom line: Test your work at every step.
