Possible NPE in internal MMOItemManager.removeItem()

[SmartfoxEnjoyer]

Good day,

I found a possible bug where a java.util.Map.get() throws a NPE because the key is null.

How i call it indirectly:

Code: Select all

    @Override
    public void Deactivate()
    {
        for (int i = 0; i < _amountToSpawn; i++)
        {
            var enemy = _enemy[i];
            if (enemy != null && _room.containsMMOItem(enemy)) // contained in room check passes
            {
                enemy.Target = null;
                _mmoAPI.removeMMOItem(enemy); // here
            }
        }

        if (!_saveState)
        {
            Arrays.fill(_enemy, null);
        }
    }

Inside MMOItemManager:

Code: Select all

  public void removeItem(BaseMMOItem item) {
      P3D lastPos = item.getLastPos();
      if (lastPos != null) { // null check done
         ConcurrentLinkedQueue<BaseMMOItem> q = (ConcurrentLinkedQueue)this.map.get(lastPos);
         if (q != null && !q.contains(item)) {
            lastPos = this.findItemLocation(item); // lastPos reassigned but null check is not done again
            q = (ConcurrentLinkedQueue)this.map.get(lastPos); // key == null == error
         }

         if (q != null) {
            q.remove(item);
         } else {
            throw new IllegalStateException();
         }
      }
   }

StackTrace:

Code: Select all

11:17:53,862 INFO  [pool-1-thread-2] Extensions     - {RoomExtension}: An error occurred in SimulationTask: Cannot invoke "Object.hashCode()" because "<parameter1>" is null
java.lang.NullPointerException: Cannot invoke "Object.hashCode()" because "<parameter1>" is null
        at java.base/java.util.concurrent.ConcurrentHashMap.get(Unknown Source)
        at com.smartfoxserver.v2.mmo.MMOItemManager.removeItem(MMOItemManager.java:152)
        at com.smartfoxserver.v2.mmo.MMORoom.removeMMOItem(MMORoom.java:228)
        at com.smartfoxserver.v2.api.SFSMMOApi.removeMMOItem(SFSMMOApi.java:169)
        at data.world.spawner.EnemySpawner.Deactivate(EnemySpawner.java:370)
        at manager.NodeManager.DeactivateNode(NodeManager.java:255)
        at manager.NodeManager.DecrementNodeUserCount(NodeManager.java:228)
        at manager.NodeManager.SetUserPosition(NodeManager.java:149)
        at manager.MovementManager.ProcessInput(MovementManager.java:185)
        at manager.SimulationManager$SimulationTask.run(SimulationManager.java:710)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

EDIT: in my test there is only 1 user, so multiple calls to SetUserPosition -> DecrementNodeUserCount -> Deactivate -> etc are not possible. Its just 1 single call.
I added traces inside NodeManager.Activate and NodeManager.Deactivate to prove its only being called once:

Code: Select all

12:48:41,368 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called activate node: 5
12:48:41,369 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 32 enemies
12:48:41,369 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 31 enemies
12:48:41,369 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 26 enemies
12:48:41,369 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called activate node: 9
12:48:41,369 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 25 enemies
12:48:41,370 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 24 enemies
12:48:41,370 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 24 enemies
12:48:53,932 INFO  [pool-1-thread-4] Extensions     - {__lib__}: Called DEACTIVATE node: 4
12:48:53,932 INFO  [pool-1-thread-4] Extensions     - {__lib__}: Called DEACTIVATE node: 8
12:48:54,265 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called activate node: 6
12:48:54,265 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Created 31 enemies
12:48:54,267 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Created 27 enemies
12:48:54,268 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called activate node: 10
12:48:54,270 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Created 29 enemies
12:49:11,064 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Called DEACTIVATE node: 5
12:49:11,064 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Called DEACTIVATE node: 9
12:49:11,497 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called activate node: 7
12:49:11,497 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 28 enemies
12:49:11,499 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called activate node: 11
12:49:11,499 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Created 24 enemies
12:49:27,162 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called DEACTIVATE node: 10
12:49:27,162 INFO  [pool-1-thread-1] Extensions     - {__lib__}: Called DEACTIVATE node: 11
12:49:27,830 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Called activate node: 2
12:49:27,830 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Created 28 enemies
12:49:27,832 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Created 24 enemies
12:49:27,832 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Called activate node: 3
12:49:33,262 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called DEACTIVATE node: 2
12:49:33,262 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called DEACTIVATE node: 3
12:49:33,661 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called activate node: 10
12:49:33,661 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Created 29 enemies
12:49:33,663 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Called activate node: 11
12:49:33,663 INFO  [pool-1-thread-2] Extensions     - {__lib__}: Created 24 enemies
12:49:38,772 INFO  [pool-1-thread-3] Extensions     - {__lib__}: Called DEACTIVATE node: 10
12:49:38,773 INFO  [pool-1-thread-3] Extensions     - {RoomExtension}: An error occurred in SimulationTask: Cannot invoke "Object.hashCode()" because "<parameter1>" is null
java.lang.NullPointerException: Cannot invoke "Object.hashCode()" because "<parameter1>" is null
        at java.base/java.util.concurrent.ConcurrentHashMap.get(Unknown Source)
        at com.smartfoxserver.v2.mmo.MMOItemManager.removeItem(MMOItemManager.java:152)
        at com.smartfoxserver.v2.mmo.MMORoom.removeMMOItem(MMORoom.java:228)
        at com.smartfoxserver.v2.api.SFSMMOApi.removeMMOItem(SFSMMOApi.java:169)
        at data.world.spawner.EnemySpawner.Deactivate(EnemySpawner.java:370)
        at manager.NodeManager.DeactivateNode(NodeManager.java:260)
        at manager.NodeManager.DecrementNodeUserCount(NodeManager.java:230)
        at manager.NodeManager.SetUserPosition(NodeManager.java:151)
        at manager.MovementManager.ProcessInput(MovementManager.java:185)
        at manager.SimulationManager$SimulationTask.run(SimulationManager.java:710)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

And it seems to work fine for a while, untill the error suddenly happens...
Thanks and cheers.

[SmartfoxEnjoyer]

Also im using java 21 LTS, maybe in older java version i had this same error before, but starting from Java 14 apparently the error became more vebose.

My old email from a year ago, refernced this same error i believe, wow! If this is it im so glad i could finally find out why!
https://github.com/TeamSailEnthusiast/P ... m/NPEStack

I sent an email on 27/01/2025.
That stack trace from 2025 was happening in a different system where i was also calling _mmoAPI.removeMMOItem().
But back then the error was not as verbose because i was using Java 11.

Now we get the exact same stacktrace pointing to ConcurretHasMap.get() and it tells us the key == null.

[SmartfoxEnjoyer]

OpenJDK 21 source:
https://github.com/openjdk/jdk/blob/mas ... shMap.java

Code: Select all

its at line 951 here, but that pretty much confirms it?
   /**
     * Returns the value to which the specified key is mapped,
     * or {@code null} if this map contains no mapping for the key.
     *
     * <p>More formally, if this map contains a mapping from a key
     * {@code k} to a value {@code v} such that {@code key.equals(k)},
     * then this method returns {@code v}; otherwise it returns
     * {@code null}.  (There can be at most one such mapping.)
     *
     * @throws NullPointerException if the specified key is null
     */
    public V get(Object key) {
        Node<K,V>[] tab; Node<K,V> e, p; int n, eh; K ek;
        int h = spread(key.hashCode()); // here

[Lapo]

Hi,
thanks for reporting. We'll add it to the next patch update.
If you need it sooner we can send a pre-release. Send us an email to support@... with a reference to this topic.

I sent an email on 27/01/2025.
That stack trace from 2025 was happening in a different system where i was also calling _mmoAPI.removeMMOItem().

Where did you send the email? We've checked our in-boxes and there is no related email on that day.

Thanks

[SmartfoxEnjoyer]

Thank you Lapo, I appreciate the quick reply!

As to the old email, I sent it to the "support" email address @thiswebsitename, on 27/01/2025 according to my inbox, but its ok I think I can wait for the next patch. Thank you again!

Cheers

[SmartfoxEnjoyer]

Is it possible after I ignore these NPEs (try/catch -> do nothing), that then .setMMOItemPosition also starts malfunctioning because the MMOItemManager state is corrupted?

I will probably email tomorrow for the pre release patch just to test out if it solves some other problems, because we are now facing some other issues that seems to point to .setMMOItemPosition failing silently a lot of times, but after some time of repeated calls to .setMMOItemPosition sometimes it does then suddenly work again and enemies appear on client side as expected.

Cheers

[SmartfoxEnjoyer]

Follow up on todays earlier problem..

I’ve been chasing a number of other MMO/AOI-related issues today, but I now strongly suspect they are all downstream effects of the same removeItem() NPE discussed earlier.

Specifically, when MMOItemManager.removeItem() throws (due to findItemLocation() returning null), the call stack aborts and the rest of MMORoom.removeMMOItem() does not execute:

Code: Select all

SFSMMOApi.removeMMOItem() -> MMORoom.removeMMOItem():

public void removeMMOItem(BaseMMOItem item) {
    this.itemsManager.removeItem(item);          // exception here
    this.updateManager.addItemToUpdate(item);    // never reached
    this.itemsById.remove(item.getId());         // never reached
}

Because addItemToUpdate() is skipped, no AOI leave/update is ever queued for clients that currently have the item in their AOI. This leaves clients with stale state (they still believe the item exists), which later causes desync issues when the same MMOItem instance is reused (pooling).

This explains symptoms such as:

Items not despawning on some clients

Reused items not appearing when re-added near their previous location

“Ghost” MMOItems existing server-side but not visible client-side

My current workaround

As a temporary mitigation, I force an AOI update before removing the item:

Code: Select all

// move item far outside any player AOI
_mmoAPI.setMMOItemPosition(item, _farAwayPosition, _room);

// then remove it
_mmoAPI.removeMMOItem(item);

This reliably resolves the issues, because moving the item triggers AOI leave events for affected clients before removeItem() is called. Even if removeItem() later throws, the clients have already been correctly updated.

This makes me believe the AOI system itself is working correctly, but missing AOI leave updates can occur whenever removeItem() throws and prevents updateManager.addItemToUpdate() from running.

Once the removeItem() NPE fix is released, I suspect these secondary issues will disappear as well - but I wanted to share this in case it helps validate the impact of the original bug.

Cheers

[Lapo]

I will probably email tomorrow for the pre release patch just to test out if it solves some other problems,

Yes, I think this would be best so if other related issues persist we can fix them out before releasing an update.

Thanks