A college would often be within its rights to take action against a student for misconduct that took place off-campus, depending on what the nature of that misconduct was. If it was, say, it would fall under Title IX, sexual harassment, sexual misconduct, if say it would be conduct of a similar nature, but would not fall under the school's Title IX policy, or say if it's conduct of a different nature, general disciplinary in nature, and say it would fall into the school's code of conduct.
Schools have various policies as to what would to use to address such misconduct that took place off-campus. It would also depend on where this took place off-campus. Did it take place in an off-campus apartment? Did it take place in a fraternity house, a sorority house? There are very fact-specific and nuance consideration, but in many instances, the school would be within its rights to pursue such action against a student.
Being accused of any kind of misconduct is incredibly serious. It can have tremendous short and long-term consequences. You need to have an experienced attorney advisor in your corner from as early as possible in the process, to help you understand and navigate the process.