Is plugin the only way when I want to extended function?

Can I use my own REST api deployed in another server instead of plugin in openfire server?

What do you want to do?

If you want to create a REST API to use the existing xmpp management API or use JMX your fine.

If you want to extend the functionality of Openfire or retrieve other internal values you need to write a plugin (or modify the Openfire code).