Solution:

Given:
- A person has 10 friends of whom six are relatives
- He wants to invite 5 guests such that 3 of them are his relatives.

To find: Total number of ways in which he can invite them.

We can solve this problem using combinations, which is a technique used in probability theory and combinatorics.

Combinations:

Combinations are the ways of selecting objects from a set without regard to the order of the objects. In other words, combinations are the ways of selecting a subset of objects from a larger set.

The formula for combinations is:

nCr = n!/(r!(n-r)!)

Where,
n = total number of objects
r = number of objects to be selected
! = factorial (product of all positive integers up to that number)

Steps to solve the problem:

Step 1: Finding the number of ways to select 3 relatives from 6 relatives
- Here, the person has 6 relatives and he wants to invite 3 of them.
- So, the number of ways to select 3 relatives from 6 relatives is given by 6C3.
- Using the formula, 6C3 = 6!/(3!(6-3)!) = 20.

Step 2: Finding the number of ways to select 2 non-relatives from 4 non-relatives
- Here, the person has 10 friends of whom 6 are relatives. So, he has 4 non-relatives.
- He wants to invite 5 guests such that 3 of them are his relatives. So, he needs to select 2 non-relatives from 4 non-relatives.
- The number of ways to select 2 non-relatives from 4 non-relatives is given by 4C2.
- Using the formula, 4C2 = 4!/(2!(4-2)!) = 6.

Step 3: Finding the total number of ways to invite 5 guests with 3 relatives and 2 non-relatives
- From Step 1, we know that there are 20 ways to select 3 relatives from 6 relatives.
- From Step 2, we know that there are 6 ways to select 2 non-relatives from 4 non-relatives.
- To find the total number of ways to invite 5 guests with 3 relatives and 2 non-relatives, we need to multiply the number of ways to select 3 relatives with the number of ways to select 2 non-relatives.
- So, the total number of ways to invite 5 guests with 3 relatives and 2 non-relatives is given by 20 * 6 = 120.

Therefore, the person can invite 5 guests with 3 relatives and 2 non-relatives in 120 ways.