When performing a behavioral simulation of the RS232 UART IP core from Qsys, a terminal emulation model is simulated to provide user-defined RS232 input and output. This is documented in the Embedded Peripheral user guide in section <UART core>.
http://www.altera.com/literature/ug/ug_embedded_ip.pdf
It is not possible to use the serial RX and TX pins directly in behavioral simulation, for such tests as loopback, or connecting to other RS232 devices.
This limitation does not affect timing simulation, which will accurately model the RX and TX pins.
Simulation support for future versions of UART IPs may be included in future version. If you are interested in this functionality please submit a Service Request via MySupport to register your interest.