Oct 30 18:34:34.301983 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 18:34:34.301992 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 18:34:34.301999 ip-10-0-128-38 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 18:34:34.302220 ip-10-0-128-38 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 18:34:44.416889 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 18:34:44.416906 ip-10-0-128-38 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cafc8689f743478b848f1ed79bb9faac -- Oct 30 18:36:43.251409 ip-10-0-128-38 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:36:43.675468 ip-10-0-128-38 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:43.675468 ip-10-0-128-38 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:36:43.675468 ip-10-0-128-38 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:43.675468 ip-10-0-128-38 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:43.680440 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680366 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.680440 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680440 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680444 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680447 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680450 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680452 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680455 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680458 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680460 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680463 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680465 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680468 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680470 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680473 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680476 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680478 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680481 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680484 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680487 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680499 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.680504 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680502 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680504 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680507 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680512 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680516 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680520 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680523 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680526 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680529 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680531 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680534 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680537 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680539 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680542 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680544 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680547 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680550 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680553 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680555 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.680976 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680557 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680560 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680562 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680566 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680568 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680571 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680573 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680576 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680578 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680580 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680583 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680585 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680588 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680597 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680600 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680603 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680606 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680608 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680611 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680613 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.681436 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680616 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680618 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680621 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680623 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680626 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680628 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680631 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680634 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680636 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680639 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680648 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680651 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680653 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680656 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680658 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680661 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680663 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680666 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680669 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.681930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680671 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680674 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680676 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680679 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680681 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680684 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680687 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680695 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680698 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680701 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680703 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680706 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680708 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680711 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680714 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680716 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680719 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680721 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680724 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.682395 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680726 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680729 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680731 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680734 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680736 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680741 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680745 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680749 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.680752 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682487 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682496 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682499 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682502 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682505 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682508 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682511 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682513 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682516 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682518 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682522 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.682833 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682526 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682529 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682532 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682534 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682537 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682539 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682542 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682545 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682548 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682550 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682553 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682555 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682558 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682561 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682563 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682566 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682569 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682572 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682575 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682577 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.683323 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682580 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682582 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682585 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682588 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682590 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682593 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682595 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682599 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682602 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682605 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682609 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682611 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682614 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682616 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682619 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682622 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682624 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682627 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682629 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682632 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.683783 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682634 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682636 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682639 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682641 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682644 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682646 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682649 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682652 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682656 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682659 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682661 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682664 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682666 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682669 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682672 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682674 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682677 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682679 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682682 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.684260 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682684 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682687 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682689 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682693 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682695 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682698 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682700 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682703 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682706 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682708 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682711 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682713 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682716 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682719 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682721 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682723 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682726 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682728 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682731 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682733 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.684738 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682736 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682741 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682744 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682747 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682749 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682752 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682754 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682757 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682759 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682762 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682764 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682766 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682769 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682771 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682774 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.682776 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683255 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683261 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683264 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683266 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.685355 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683269 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683272 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683275 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683277 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683280 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683282 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683285 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683287 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683290 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683292 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683295 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683298 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683300 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683304 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683307 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683310 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683312 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683315 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683318 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.686078 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683320 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683323 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683325 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683328 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683331 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683333 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683336 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683339 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683341 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683344 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683346 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683349 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683351 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683354 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683357 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683359 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683362 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683364 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683367 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683369 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.686554 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683372 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683374 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683377 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683379 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683382 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683385 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683388 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683391 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683393 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683395 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683398 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683400 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683403 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683405 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683408 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683410 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683413 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683416 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683420 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.687018 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683424 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683427 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683430 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683434 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683436 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683439 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683441 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683445 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683449 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683451 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683454 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683456 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683458 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683461 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683463 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683466 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683468 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683471 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683474 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.687493 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683476 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683479 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683481 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683484 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683486 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683488 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683491 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683493 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683496 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683498 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683501 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683503 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683505 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683508 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683510 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683512 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683515 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683517 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683520 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683523 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.687938 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683525 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683528 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683531 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683533 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.683536 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684580 2579 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684594 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684599 2579 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684604 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684608 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684611 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684615 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684619 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684622 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684625 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684628 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684632 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684635 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684637 2579 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684640 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684643 2579 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684646 2579 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684649 2579 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:36:43.688409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684653 2579 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684656 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684659 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684662 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684665 2579 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684669 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684671 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684674 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684677 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684680 2579 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684683 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684686 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684689 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684692 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684696 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684699 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684702 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684705 2579 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684708 2579 flags.go:64] FLAG: --enable-server="true" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684710 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684717 2579 flags.go:64] FLAG: --event-burst="100" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684720 2579 flags.go:64] FLAG: --event-qps="50" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684723 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684726 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:36:43.689018 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684728 2579 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684733 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684735 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684738 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684741 2579 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684745 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684748 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684751 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684754 2579 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684756 2579 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684759 2579 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684762 2579 flags.go:64] FLAG: --feature-gates="" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684767 2579 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684770 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684773 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684776 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684779 2579 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684782 2579 flags.go:64] FLAG: --help="false" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684785 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684788 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684791 2579 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684796 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684799 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:36:43.689667 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684802 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684805 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684808 2579 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684811 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684814 2579 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684817 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684820 2579 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684823 2579 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684826 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684828 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684831 2579 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684834 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684837 2579 flags.go:64] FLAG: --lock-file="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684839 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684842 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684845 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684851 2579 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684854 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684857 2579 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684859 2579 flags.go:64] FLAG: --logging-format="text" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684862 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684867 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684870 2579 flags.go:64] FLAG: --manifest-url="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684872 2579 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684877 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:36:43.690227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684880 2579 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684884 2579 flags.go:64] FLAG: --max-pods="110" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684886 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684889 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684892 2579 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684895 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684898 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684901 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684904 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684911 2579 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684914 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684917 2579 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684920 2579 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684923 2579 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684925 2579 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684928 2579 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684931 2579 flags.go:64] FLAG: --port="10250" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684934 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684937 2579 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f0aacc8f40e0bf79" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684940 2579 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684942 2579 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684945 2579 flags.go:64] FLAG: --register-node="true" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684948 2579 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684952 2579 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684954 2579 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.684957 2579 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:36:43.690807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685486 2579 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685502 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685512 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685517 2579 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685522 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685528 2579 flags.go:64] FLAG: --runonce="false" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685533 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685538 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685550 2579 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685555 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685560 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685565 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685570 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685582 2579 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685586 2579 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685591 2579 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685595 2579 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685604 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685609 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685614 2579 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685618 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685639 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685644 2579 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685648 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685661 2579 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:36:43.691437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685666 2579 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685670 2579 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685675 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685680 2579 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685684 2579 flags.go:64] FLAG: --v="2" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685713 2579 flags.go:64] FLAG: --version="false" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685719 2579 flags.go:64] FLAG: --vmodule="" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685764 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.685774 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686262 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686273 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686277 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686281 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686284 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686287 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686289 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686292 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686294 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686297 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686300 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686303 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.692055 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686306 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686308 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686311 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686316 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686320 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686323 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686326 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686329 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686332 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686335 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686337 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686340 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686342 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686345 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686347 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686350 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686352 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686355 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686357 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.692568 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686360 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686362 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686365 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686368 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686372 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686374 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686377 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686379 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686382 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686385 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686388 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686390 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686393 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686395 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686398 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686400 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686403 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686406 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686408 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.693010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686411 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686415 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686418 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686420 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686423 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686425 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686428 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686431 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686433 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686435 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686438 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686440 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686443 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686446 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686449 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686452 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686454 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686458 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686460 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686463 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.693465 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686465 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686468 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686471 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686473 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686476 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686478 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686481 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686483 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686486 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686488 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686492 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686495 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686498 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686502 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686505 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686508 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686511 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686513 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686516 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.693969 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686519 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686521 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686523 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686526 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686528 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686531 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686534 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686537 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686539 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686542 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686546 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686548 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686551 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686554 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686556 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686559 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.694431 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686562 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.686591 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686632 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686636 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686639 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686642 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686645 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686648 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686651 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686653 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686661 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686664 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686666 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686669 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686671 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686674 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.694813 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686676 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686679 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686681 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686684 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686687 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686689 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686692 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686694 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686697 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686700 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686702 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686706 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686709 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686711 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686714 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686716 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686719 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686721 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686724 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.695245 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686727 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686729 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686732 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686735 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686737 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686740 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686743 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686746 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686749 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686752 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686755 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686757 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686760 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686762 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686765 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686769 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686772 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686774 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686777 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.695695 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686779 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686782 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686784 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686787 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686789 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686793 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686795 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686798 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686800 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686803 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686805 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686808 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686811 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686813 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686816 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686818 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686821 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686823 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686826 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686829 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.696147 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686833 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686836 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686839 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686842 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686844 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686847 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686850 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686852 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686855 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686857 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686859 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686862 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686864 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686867 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686870 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686872 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686876 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686881 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686884 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.696595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686886 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686889 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686892 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686895 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686897 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686901 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686903 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686906 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686908 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686911 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686913 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686915 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686918 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686921 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.686924 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.693483 2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.693500 2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.694174 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.694181 2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694242 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.697076 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694247 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694251 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694254 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694257 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694260 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694263 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694265 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694268 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694271 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694273 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694275 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694278 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694281 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694284 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694286 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694288 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694291 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694294 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694297 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.697564 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694299 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694302 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694304 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694307 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694310 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694312 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694315 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694317 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694320 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694324 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694327 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694329 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694331 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694334 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694337 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694339 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694342 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694344 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694347 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.698011 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694349 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694352 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694354 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694357 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694359 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694362 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694364 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694367 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694369 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694372 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694374 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694377 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694379 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694382 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694384 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694387 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694390 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694392 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694395 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694397 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694400 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.698473 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694402 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694405 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694409 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694412 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694414 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694417 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694419 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694422 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694424 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694427 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694429 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694432 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694436 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694440 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694443 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694446 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694449 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694453 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694455 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694458 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.699010 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694461 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694463 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694466 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694469 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694471 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694474 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694476 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694479 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694481 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694484 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694488 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694492 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694496 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694498 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694501 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694504 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694506 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694509 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694511 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.699581 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694514 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694517 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694520 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694522 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694525 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694527 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694529 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.694557 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694691 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694695 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694698 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694700 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694703 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694705 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694708 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694711 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.700026 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694713 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694716 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694720 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694723 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694726 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694729 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694732 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694735 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694737 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694739 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694742 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694744 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694747 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694750 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694753 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694755 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694758 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694760 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694763 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.700428 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694765 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694768 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694770 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694773 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694776 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694778 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694781 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694783 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694785 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694788 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694790 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694793 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694796 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694798 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694800 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694803 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694806 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694808 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694811 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.700876 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694813 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694816 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694819 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694822 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694825 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694828 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694830 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694833 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694836 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694838 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694841 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694843 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694845 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694848 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694850 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694853 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694856 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694858 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694861 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694863 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.701325 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694866 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694868 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694871 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694873 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694876 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694878 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694881 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694883 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694886 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694888 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694891 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694893 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694895 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694899 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694903 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694906 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694909 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694912 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694915 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.701777 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694917 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694920 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694923 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694926 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694928 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694931 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694933 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694936 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694938 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694941 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694943 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694946 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694948 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694951 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694953 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694955 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694958 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694961 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694963 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694965 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.702238 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.694968 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.694991 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695021 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695026 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695029 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695032 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695035 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695038 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695041 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695043 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695046 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695048 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695051 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695053 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695056 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695059 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:43.702704 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695062 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695065 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695067 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695070 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695081 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695084 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695086 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695088 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695109 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695112 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695114 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695117 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695120 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695122 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695124 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695127 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695129 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695132 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695136 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695140 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:43.703065 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695143 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695146 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695148 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695151 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695153 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695156 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695158 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695161 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695163 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695165 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695168 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695170 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695174 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695177 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695179 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695182 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695184 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695187 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695189 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695192 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:43.703550 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695195 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695198 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695200 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695202 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695205 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695207 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695210 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695212 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695215 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695217 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695220 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695223 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695225 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695228 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695230 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695234 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695237 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695242 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695245 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:43.703997 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695247 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695250 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695253 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695256 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695258 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695261 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695264 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695267 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695270 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695272 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695275 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695277 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695279 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695282 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695284 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695287 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695289 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695291 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695294 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:43.704542 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695296 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695299 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695301 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695304 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695306 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695310 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695312 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695315 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695318 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695320 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695322 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695326 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695328 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:43.695331 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.695383 2579 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.696841 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.698500 2579 server.go:1014] "Starting client certificate rotation" Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.698596 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:36:43.704977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.698630 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:36:43.705402 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.700599 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:36:43.705402 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.701756 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:36:43.713512 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.713495 2579 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:36:43.715722 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.715706 2579 log.go:25] "Validated CRI v1 image API" Oct 30 18:36:43.716577 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.716556 2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:36:43.719160 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.719142 2579 fs.go:135] Filesystem UUIDs: map[6cff51bd-09d2-4299-9ad1-9febfd51eb5b:/dev/nvme0n1p4 70237958-2b29-40d7-9b22-f2916fd4c3e8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Oct 30 18:36:43.719219 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.719159 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:36:43.723247 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.723141 2579 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:43.722649813 +0000 UTC m=+0.350494987 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096437 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec290e98ecedbf2334c34433c794980b SystemUUID:ec290e98-eced-bf23-34c3-4433c794980b BootID:cafc8689-f743-478b-848f-1ed79bb9faac Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:cb:dd:7f:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:cb:dd:7f:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:c9:2d:22:35:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:36:43.723891 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.723881 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:36:43.723963 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.723952 2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:36:43.727479 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.727452 2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:36:43.727600 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.727480 2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-38.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:36:43.729062 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.729052 2579 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:36:43.729116 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.729065 2579 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:36:43.729116 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.729078 2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:36:43.729883 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.729873 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:36:43.730678 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.730668 2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:36:43.734896 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.734884 2579 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:36:43.735012 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735002 2579 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:36:43.735930 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735921 2579 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:36:43.735961 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735933 2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:36:43.735961 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735948 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:36:43.735961 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735958 2579 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:36:43.736051 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.735966 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:36:43.736288 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.736276 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:36:43.738424 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.738406 2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:36:43.743684 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.743672 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:36:43.744425 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744414 2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:36:43.744458 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744444 2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:36:43.744603 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744593 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:36:43.744630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744608 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:36:43.744630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744614 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:36:43.744630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744620 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:36:43.744630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744627 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:36:43.744727 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.744634 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:36:43.745169 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745161 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:36:43.745169 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745169 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:36:43.745232 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745176 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:36:43.745232 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745181 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:36:43.745232 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745195 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:36:43.745232 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745204 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:36:43.745928 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745920 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:36:43.745957 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.745929 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:36:43.749700 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.749686 2579 server.go:1263] "Started kubelet" Oct 30 18:36:43.749797 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.749765 2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:36:43.749899 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.749776 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:36:43.749899 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.749855 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:36:43.750354 ip-10-0-128-38 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:36:43.751177 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.751160 2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:36:43.753605 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.753581 2579 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:36:43.756478 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.756458 2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:36:43.759113 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.759081 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:36:43.760157 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760141 2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:36:43.760255 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.760231 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:43.760855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760837 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:36:43.760855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760840 2579 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:36:43.760952 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760864 2579 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:36:43.760952 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760946 2579 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:36:43.760952 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760951 2579 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:36:43.761033 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.760989 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:36:43.761033 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761000 2579 factory.go:55] Registering systemd factory Oct 30 18:36:43.761033 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761006 2579 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:36:43.761238 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761222 2579 factory.go:153] Registering CRI-O factory Oct 30 18:36:43.761270 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761244 2579 factory.go:223] Registration of the crio container factory successfully Oct 30 18:36:43.761270 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761262 2579 factory.go:103] Registering Raw factory Oct 30 18:36:43.761325 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.761276 2579 manager.go:1199] Started watching for new ooms in manager Oct 30 18:36:43.762146 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.762130 2579 manager.go:319] Starting recovery of all containers Oct 30 18:36:43.768482 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.768452 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:36:43.772987 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.772967 2579 manager.go:324] Recovery completed Oct 30 18:36:43.774335 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.774316 2579 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Oct 30 18:36:43.777071 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.777053 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.778713 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.778697 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.778781 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.778726 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.778781 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.778736 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.779243 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779231 2579 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:36:43.779243 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779241 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.779316 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779242 2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:36:43.779316 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779292 2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:36:43.779832 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779818 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.779884 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779842 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.779884 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.779852 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.781577 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.781562 2579 policy_none.go:50] "Start" Oct 30 18:36:43.781577 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.781579 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:36:43.781668 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.781594 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:36:43.784211 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.784200 2579 policy_none.go:44] "Start" Oct 30 18:36:43.818784 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.818654 2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:36:43.822719 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.822705 2579 manager.go:341] "Starting Device Plugin manager" Oct 30 18:36:43.823761 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.823741 2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.823764 2579 server.go:87] "Starting device plugin registration server" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.824030 2579 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.824043 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.824162 2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.824288 2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.824298 2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.824679 2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.824723 2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.841743 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.841764 2579 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.841782 2579 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:36:43.847866 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.841809 2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 18:36:43.882820 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.882792 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-38.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:36:43.883719 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.882746 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-38.us-west-2.compute.internal.18e36285dfdc8eca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-38.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-38.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:43.749535434 +0000 UTC m=+0.377380609,LastTimestamp:2026-10-30 18:36:43.749535434 +0000 UTC m=+0.377380609,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-38.us-west-2.compute.internal,}" Oct 30 18:36:43.925131 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.925111 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.925905 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.925854 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.925905 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.925889 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.925905 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.925903 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.926042 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.925924 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:43.927039 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.927018 2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-38.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:36:43.931795 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.931771 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-38.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 18:36:43.931887 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.931817 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:36:43.932080 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.932002 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-38.us-west-2.compute.internal.18e36285e199c2b3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-38.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-38.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-38.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:43.778712243 +0000 UTC m=+0.406557418,LastTimestamp:2026-10-30 18:36:43.778712243 +0000 UTC m=+0.406557418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-38.us-west-2.compute.internal,}" Oct 30 18:36:43.942011 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.941990 2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal"] Oct 30 18:36:43.942109 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.942047 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.942756 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.942731 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.942833 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.942762 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.942833 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.942777 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.944924 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.944910 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.945072 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945058 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:43.945142 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945084 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.945560 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945543 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.945650 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945569 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.945650 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945584 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.945650 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945608 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.945650 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945632 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.945650 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.945647 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.947612 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.947595 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:43.947710 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.947620 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:43.948296 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.948282 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:43.948368 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.948305 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:43.948368 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.948319 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:43.972862 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.972844 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:36:43.972923 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:43.972906 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kzjsd" Oct 30 18:36:43.973780 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.973758 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:43.980256 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:43.980236 2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.027448 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.027431 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:44.027560 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.027544 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:44.062143 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.062123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.062218 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.062149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.062218 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.062167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.073755 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.073735 2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.073755 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.073753 2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-38.us-west-2.compute.internal\": node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.163234 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.163234 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163236 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.163357 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.163357 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163312 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.163357 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163318 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.163357 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.163318 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.190312 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.190292 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.277098 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.277070 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.283705 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.283684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:44.290378 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.290356 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.391429 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.391406 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.492169 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.492123 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.592959 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.592935 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.693999 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.693970 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.698159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.698140 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:36:44.698350 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.698267 2579 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a000931687ba14c969afc60d3cadd041-7b43774db3a78fc8.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-128-38.us-west-2.compute.internal.18e36285e19a2c9d\": read tcp 10.0.128.38:55700->18.227.231.83:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-128-38.us-west-2.compute.internal.18e36285e19a2c9d default 5195 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-38.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-38.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-38.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:43 +0000 UTC,LastTimestamp:2026-10-30 18:36:43.942783111 +0000 UTC m=+0.570628291,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-38.us-west-2.compute.internal,}" Oct 30 18:36:44.698404 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.698305 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 18:36:44.698404 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.698305 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 18:36:44.795142 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.795058 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.895417 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.895380 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:44.974576 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.974552 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:44.975503 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.975483 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:44.975592 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.975518 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:44.975592 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.975532 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:44.977648 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.977624 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:43 +0000 UTC" deadline="2028-05-30 20:29:27.185912728 +0000 UTC" Oct 30 18:36:44.977689 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:44.977647 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13873h52m42.208268818s" Oct 30 18:36:44.996181 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:44.996161 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:45.096531 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.096487 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:36:45.137905 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.137888 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:45.160580 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.160560 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:45.194669 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.194645 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:45.220616 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.220595 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:36:45.221552 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.221538 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:36:45.283140 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.283120 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:36:45.304930 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:45.304902 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bec2dc02e02013dddc74f8c61aeafbc.slice/crio-30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0 WatchSource:0}: Error finding container 30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0: Status 404 returned error can't find the container with id 30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0 Oct 30 18:36:45.305442 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:45.305423 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6479c60b669fe79980e9a61328bda9.slice/crio-a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2 WatchSource:0}: Error finding container a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2: Status 404 returned error can't find the container with id a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2 Oct 30 18:36:45.309239 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.309225 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:36:45.736582 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.736549 2579 apiserver.go:52] "Watching apiserver" Oct 30 18:36:45.760551 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.760508 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:36:45.846108 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.846042 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"1c6479c60b669fe79980e9a61328bda9","Type":"ContainerStarted","Data":"a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2"} Oct 30 18:36:45.847192 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.847171 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:45.847321 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.847300 2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:36:45.847653 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.847634 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerStarted","Data":"30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0"} Oct 30 18:36:45.847707 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.847671 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-25pf7","kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-4xjt2","openshift-multus/multus-additional-cni-plugins-jnv56","openshift-network-diagnostics/network-check-target-wnj4b","kube-system/global-pull-secret-syncer-plzrc","openshift-dns/node-resolver-c96l5","openshift-network-operator/iptables-alerter-rpdb6","openshift-ovn-kubernetes/ovnkube-node-dbz7h","kube-system/konnectivity-agent-zllgp","openshift-image-registry/node-ca-kxg89","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-sndlp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh"] Oct 30 18:36:45.851778 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.851760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.853982 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.853954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:45.854076 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.854039 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:45.856097 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.856073 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.858655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.858508 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.860765 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.860744 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:45.860843 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.860822 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:45.862969 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.862948 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.863038 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.863018 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:45.865430 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.865404 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.867752 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.867726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.869923 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.869905 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.872191 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.872272 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:45.872272 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872209 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:45.872272 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:45.872272 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872355 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872379 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872449 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.872655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872478 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.872655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.872655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cll7\" (UniqueName: \"kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.872655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872771 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872653 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.872771 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872693 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.872771 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.872771 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872800 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872843 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.872980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.873037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6rd6\" (UniqueName: \"kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.873448 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873049 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873448 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98jlw\" (UniqueName: \"kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.873515 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.873555 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48fqm\" (UniqueName: \"kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:45.873597 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.873584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.875477 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.875418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:45.877370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.877347 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:45.908451 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.908430 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:36:45.908535 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.908486 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:36:45.908673 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.908659 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-djndp\"" Oct 30 18:36:45.911447 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.911428 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k6xhn\"" Oct 30 18:36:45.959655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.959633 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:36:45.959742 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.959687 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9txh9" Oct 30 18:36:45.962319 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.962305 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:36:45.974180 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.974250 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.974250 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rd6\" (UniqueName: \"kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.974250 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.974377 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.974428 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:45.974467 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974447 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.974510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974476 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974601 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwsp\" (UniqueName: \"kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974601 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.974573 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:45.974666 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.974666 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.974638 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:46.474616736 +0000 UTC m=+3.102461930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48fqm\" (UniqueName: \"kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.974775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974872 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv87d\" (UniqueName: \"kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.974980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975037 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975017 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975111 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975188 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975188 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975235 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975264 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975306 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpjb\" (UniqueName: \"kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975420 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.975456 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvck9\" (UniqueName: \"kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975611 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975629 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:45.975694 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975696 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975851 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98jlw\" (UniqueName: \"kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.975916 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.975979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976005 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.976050 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976071 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:45.976114 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:46.476084174 +0000 UTC m=+3.103929341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976168 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976219 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976297 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976331 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976351 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976423 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cll7\" (UniqueName: \"kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976481 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976490 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg47z\" (UniqueName: \"kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:45.976607 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976750 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976820 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976916 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976892 2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.976979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977085 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.977330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:45.977805 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977215 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:45.977805 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.977241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:45.980199 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.980181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:45.980260 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:45.980225 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:46.007560 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.007541 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:46.013378 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013358 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:36:46.013461 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013404 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:36:46.013523 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013471 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.013523 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013507 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:36:46.013523 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013511 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.013664 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013539 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:36:46.013664 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013586 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.013664 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013611 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.013747 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013721 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.013792 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013773 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.013841 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013777 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.013894 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013850 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:36:46.013894 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013852 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vmn22\"" Oct 30 18:36:46.013953 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013893 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:36:46.013953 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.013893 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:36:46.014078 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014051 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.014145 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014079 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:36:46.014145 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014082 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.014145 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014132 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:36:46.014145 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014138 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:36:46.014284 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014151 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j7txc\"" Oct 30 18:36:46.014284 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014156 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:46.014284 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014225 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.014432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014284 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:46.014432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014291 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:36:46.014432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014342 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-thsk7\"" Oct 30 18:36:46.014432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014400 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sslwz\"" Oct 30 18:36:46.014432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014429 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:36:46.014601 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014492 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:36:46.014601 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014563 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-p85r8\"" Oct 30 18:36:46.014700 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014684 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:46.014831 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.014815 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:46.016697 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.016678 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:46.026011 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.025998 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w6mkd\"" Oct 30 18:36:46.026051 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.026000 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-h66ck\"" Oct 30 18:36:46.034425 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.034408 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:46.034425 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.034425 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:46.034553 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.034434 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:46.034553 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.034472 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:46.53446168 +0000 UTC m=+3.162306884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:46.036758 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.036733 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48fqm\" (UniqueName: \"kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:46.036844 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.036786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6rd6\" (UniqueName: \"kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:46.036844 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.036819 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98jlw\" (UniqueName: \"kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:46.037013 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.036997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cll7\" (UniqueName: \"kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:46.078202 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.078290 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078290 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078290 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078290 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078279 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078313 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078323 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078323 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg47z\" (UniqueName: \"kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078340 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078421 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078432 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078467 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.078469 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078517 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078549 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078553 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078635 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078703 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078702 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078755 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078777 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078801 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078804 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079083 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078827 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwsp\" (UniqueName: \"kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078982 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078937 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.078999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079094 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079047 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079146 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079159 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079198 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.079646 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv87d\" (UniqueName: \"kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079307 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079314 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079760 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079940 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079978 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080037 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080077 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080091 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.080506 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.079983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080239 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpjb\" (UniqueName: \"kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080095 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080412 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080427 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvck9\" (UniqueName: \"kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080432 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080630 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.081327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080658 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080724 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080776 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.080904 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.081152 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.082064 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.081310 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.083010 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.082866 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.083010 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.082962 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:46.139790 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.139735 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg47z\" (UniqueName: \"kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.140410 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.140388 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv87d\" (UniqueName: \"kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.140823 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.140798 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwsp\" (UniqueName: \"kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.141856 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.141836 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvck9\" (UniqueName: \"kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.143783 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.143761 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpjb\" (UniqueName: \"kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.163988 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.163961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c96l5" Oct 30 18:36:46.171265 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.171179 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:36:46.173940 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.173838 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3aa381_aaa3_4f19_9df0_dbc93b3b1fe7.slice/crio-efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39 WatchSource:0}: Error finding container efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39: Status 404 returned error can't find the container with id efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39 Oct 30 18:36:46.179565 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.179544 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5e5aef_db22_432b_9bfc_64231443a33c.slice/crio-b49125e8a141b3981d346b4a07a6cb1d78be2710d7b15fe1c96091372323c16a WatchSource:0}: Error finding container b49125e8a141b3981d346b4a07a6cb1d78be2710d7b15fe1c96091372323c16a: Status 404 returned error can't find the container with id b49125e8a141b3981d346b4a07a6cb1d78be2710d7b15fe1c96091372323c16a Oct 30 18:36:46.197658 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.197639 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-25pf7" Oct 30 18:36:46.199963 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.199944 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:36:46.205858 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.205837 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cc7620_49dc_4d5a_8343_411e0d17309d.slice/crio-9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766 WatchSource:0}: Error finding container 9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766: Status 404 returned error can't find the container with id 9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766 Oct 30 18:36:46.208343 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.208236 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:46.209390 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.209366 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf1866b_1e63_4886_830d_c8645a8e5ff5.slice/crio-779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e WatchSource:0}: Error finding container 779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e: Status 404 returned error can't find the container with id 779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e Oct 30 18:36:46.215250 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.215230 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:36:46.217378 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.217359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:36:46.217595 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.217578 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7268393d_9be4_4e1c_87ff_c9ac7cfd7072.slice/crio-fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97 WatchSource:0}: Error finding container fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97: Status 404 returned error can't find the container with id fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97 Oct 30 18:36:46.221430 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.221413 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:36:46.223244 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.223227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:36:46.227160 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.227084 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a02e6e_236c_4782_95af_7e99fa284781.slice/crio-b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193 WatchSource:0}: Error finding container b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193: Status 404 returned error can't find the container with id b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193 Oct 30 18:36:46.228192 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.228165 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e82816_cfc3_413b_a254_47dd3a2535ea.slice/crio-26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0 WatchSource:0}: Error finding container 26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0: Status 404 returned error can't find the container with id 26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0 Oct 30 18:36:46.233329 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.233311 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e02e4a_2151_4339_93de_09057afd8f9d.slice/crio-af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69 WatchSource:0}: Error finding container af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69: Status 404 returned error can't find the container with id af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69 Oct 30 18:36:46.235441 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:36:46.235363 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ca3b9e_8619_467a_a705_5bcbabdd7a08.slice/crio-4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725 WatchSource:0}: Error finding container 4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725: Status 404 returned error can't find the container with id 4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725 Oct 30 18:36:46.483712 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.483663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:46.483712 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.483724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:46.483977 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.483844 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:46.483977 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.483893 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.483880365 +0000 UTC m=+4.111725527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:46.483977 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.483970 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:46.483977 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.484004 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.4839923 +0000 UTC m=+4.111837479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:46.585016 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.584981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:46.585217 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.585182 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:46.585217 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.585204 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:46.585217 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.585217 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:46.585422 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:46.585277 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.585258177 +0000 UTC m=+4.213103355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:46.851074 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.850937 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rpdb6" event={"ID":"f0e02e4a-2151-4339-93de-09057afd8f9d","Type":"ContainerStarted","Data":"af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69"} Oct 30 18:36:46.852559 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.852528 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" event={"ID":"e4e82816-cfc3-413b-a254-47dd3a2535ea","Type":"ContainerStarted","Data":"26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0"} Oct 30 18:36:46.853728 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.853702 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zllgp" event={"ID":"7268393d-9be4-4e1c-87ff-c9ac7cfd7072","Type":"ContainerStarted","Data":"fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97"} Oct 30 18:36:46.855001 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.854974 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerStarted","Data":"779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e"} Oct 30 18:36:46.856432 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.856405 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c96l5" event={"ID":"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7","Type":"ContainerStarted","Data":"efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39"} Oct 30 18:36:46.857709 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.857686 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725"} Oct 30 18:36:46.859030 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.858986 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxg89" event={"ID":"05a02e6e-236c-4782-95af-7e99fa284781","Type":"ContainerStarted","Data":"b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193"} Oct 30 18:36:46.860286 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.860262 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-25pf7" event={"ID":"f4cc7620-49dc-4d5a-8343-411e0d17309d","Type":"ContainerStarted","Data":"9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766"} Oct 30 18:36:46.861530 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.861509 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" event={"ID":"7f5e5aef-db22-432b-9bfc-64231443a33c","Type":"ContainerStarted","Data":"b49125e8a141b3981d346b4a07a6cb1d78be2710d7b15fe1c96091372323c16a"} Oct 30 18:36:46.962424 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.962388 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:45 +0000 UTC" deadline="2028-05-14 01:57:01.097558049 +0000 UTC" Oct 30 18:36:46.962424 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:46.962423 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13471h20m14.135139725s" Oct 30 18:36:47.492345 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.492309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:47.492516 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.492407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:47.492580 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.492541 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:47.492629 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.492616 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:49.492579269 +0000 UTC m=+6.120424438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:47.492976 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.492955 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:47.493047 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.493011 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:49.492995652 +0000 UTC m=+6.120840822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:47.593344 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.593307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:47.593518 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.593484 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:47.593518 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.593500 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:47.593518 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.593511 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:47.593665 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.593560 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:49.593545235 +0000 UTC m=+6.221390404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:47.842370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.842344 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:47.842507 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.842484 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:47.842979 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.842957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:47.843369 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.843181 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:47.843369 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.843269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:47.843369 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:47.843341 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:47.963570 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.963340 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:45 +0000 UTC" deadline="2028-04-04 06:54:55.032270359 +0000 UTC" Oct 30 18:36:47.963961 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:47.963574 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12516h18m7.068702481s" Oct 30 18:36:48.886655 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:48.886617 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"1c6479c60b669fe79980e9a61328bda9","Type":"ContainerStarted","Data":"fb5eb5933f9b98d5bcd27028011c0dc18a7c518c6d9331fe1fa28b23ea07e73f"} Oct 30 18:36:48.902266 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:48.902232 2579 generic.go:356] "Generic (PLEG): container finished" podID="3bec2dc02e02013dddc74f8c61aeafbc" containerID="84bc559fa0bc591105cd035863c805aa961c8b5332aaae183119cadffff4203e" exitCode=0 Oct 30 18:36:48.902529 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:48.902286 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerDied","Data":"84bc559fa0bc591105cd035863c805aa961c8b5332aaae183119cadffff4203e"} Oct 30 18:36:49.124570 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.124247 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" podStartSLOduration=4.124227592 podStartE2EDuration="4.124227592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:49.004218713 +0000 UTC m=+5.632063900" watchObservedRunningTime="2026-10-30 18:36:49.124227592 +0000 UTC m=+5.752072779" Oct 30 18:36:49.512206 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.512177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:49.512328 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.512238 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:49.512382 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.512344 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:49.512435 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.512394 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:53.512378152 +0000 UTC m=+10.140223328 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:49.512496 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.512453 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:49.512496 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.512484 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:53.51247392 +0000 UTC m=+10.140319096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:49.612904 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.612872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:49.613155 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.613076 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:49.613155 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.613115 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:49.613155 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.613127 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:49.613360 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.613181 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:53.613164316 +0000 UTC m=+10.241009493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:49.843287 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.843206 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:49.843462 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.843334 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:49.843863 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.843727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:49.843863 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.843831 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:49.844003 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.843883 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:49.844003 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:49.843950 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:49.922308 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.922276 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerStarted","Data":"4a548068e9b53ca48ed91bcd7b1154b61d2ea966fd16aa9ca96a6f36434605d3"} Oct 30 18:36:49.927548 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.927516 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="1883c9dafa809ff182d8af27e4a19c1c1e6aed06b8a5d8010d93097378db3513" exitCode=0 Oct 30 18:36:49.927920 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:49.927900 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"1883c9dafa809ff182d8af27e4a19c1c1e6aed06b8a5d8010d93097378db3513"} Oct 30 18:36:50.189113 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:50.188512 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" podStartSLOduration=5.188492056 podStartE2EDuration="5.188492056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:50.043309797 +0000 UTC m=+6.671155000" watchObservedRunningTime="2026-10-30 18:36:50.188492056 +0000 UTC m=+6.816337241" Oct 30 18:36:50.947033 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:50.946738 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zllgp" event={"ID":"7268393d-9be4-4e1c-87ff-c9ac7cfd7072","Type":"ContainerStarted","Data":"a1e57c01803322df16e90c68711e865cf0f472bd22fb830f93104c7625d6bcb7"} Oct 30 18:36:51.507803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.507771 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:51.508509 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.508487 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:51.635685 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.635639 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zllgp" podStartSLOduration=3.371938188 podStartE2EDuration="7.635625166s" totalImagesPullingTime="4.263686978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:51.065190709 +0000 UTC m=+7.693035894" watchObservedRunningTime="2026-10-30 18:36:51.635625166 +0000 UTC m=+8.263470360" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.842314 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:51.842436 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.842503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:51.842585 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.842272 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:51.843293 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:51.843213 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:51.957871 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:51.957373 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxg89" event={"ID":"05a02e6e-236c-4782-95af-7e99fa284781","Type":"ContainerStarted","Data":"c583bea6a1f51de555951d1842191ea86242427345be361230f983f8946ac879"} Oct 30 18:36:52.964498 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:52.964456 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" event={"ID":"e4e82816-cfc3-413b-a254-47dd3a2535ea","Type":"ContainerStarted","Data":"794f594c1165219311a26e8bba9a9aea935074dd4b3e6847e08594e12945e681"} Oct 30 18:36:52.964974 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:52.964588 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:36:53.544015 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.543975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:53.544187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.544045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:53.544233 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.544192 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:53.544272 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.544257 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.544239099 +0000 UTC m=+18.172084268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:53.544685 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.544657 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:53.544758 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.544720 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.544704454 +0000 UTC m=+18.172549629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:53.645071 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.645039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:53.645263 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.645189 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:53.645263 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.645209 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:53.645263 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.645222 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:53.645412 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.645281 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.645262848 +0000 UTC m=+18.273108027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:53.845728 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.845657 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:53.845888 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.845785 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:53.845961 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.845919 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:53.846029 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.846001 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:53.846087 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:53.846055 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:53.846163 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:53.846136 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:54.595004 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.594976 2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:36:54.835805 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.835713 2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:36:54.594999628Z","UUID":"bcc1c8b3-33c8-4bb4-ae9d-e8578511e499","Handler":null,"Name":"","Endpoint":""} Oct 30 18:36:54.837409 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.837378 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:36:54.837505 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.837418 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:36:54.971651 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.971402 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-25pf7" event={"ID":"f4cc7620-49dc-4d5a-8343-411e0d17309d","Type":"ContainerStarted","Data":"fee17ffcc8751e280e09308c27d3230d8e3378422fb1e0a4cb24128a6f9e23d2"} Oct 30 18:36:54.975266 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.975209 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" event={"ID":"e4e82816-cfc3-413b-a254-47dd3a2535ea","Type":"ContainerStarted","Data":"68e46f4ae0e47a1791f7a7bfaecc8828abcd8e856bf5e48cb311c13b4c1790ab"} Oct 30 18:36:54.977626 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.977595 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerStarted","Data":"ae5c26a8f8bbcd155a185728eb1097b12ac51c86445fd75862c1c18e274d3db5"} Oct 30 18:36:54.983048 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:54.983008 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c96l5" event={"ID":"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7","Type":"ContainerStarted","Data":"7927e28e3fa0d2f45cb675fc3bee4fd76fc5d2d0cdd065aa5e8f133a0c4f1f5c"} Oct 30 18:36:55.118927 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.118868 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kxg89" podStartSLOduration=5.806578967 podStartE2EDuration="11.11885126s" totalImagesPullingTime="5.312272293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:52.076582634 +0000 UTC m=+8.704427821" watchObservedRunningTime="2026-10-30 18:36:55.11885126 +0000 UTC m=+11.746696445" Oct 30 18:36:55.119676 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.119630 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-25pf7" podStartSLOduration=3.102071695 podStartE2EDuration="11.119616638s" totalImagesPullingTime="8.017544943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:55.119259693 +0000 UTC m=+11.747104879" watchObservedRunningTime="2026-10-30 18:36:55.119616638 +0000 UTC m=+11.747461823" Oct 30 18:36:55.246258 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.246208 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-c96l5" podStartSLOduration=2.7912404090000003 podStartE2EDuration="11.246189706s" totalImagesPullingTime="8.454949297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:55.245862742 +0000 UTC m=+11.873707927" watchObservedRunningTime="2026-10-30 18:36:55.246189706 +0000 UTC m=+11.874034893" Oct 30 18:36:55.844500 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.844426 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:55.844500 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.844477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:55.844991 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:55.844599 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:55.845169 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:55.845077 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:55.845271 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.845164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:55.845336 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:55.845290 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:55.988198 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.988163 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="ae5c26a8f8bbcd155a185728eb1097b12ac51c86445fd75862c1c18e274d3db5" exitCode=0 Oct 30 18:36:55.989204 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:55.989176 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"ae5c26a8f8bbcd155a185728eb1097b12ac51c86445fd75862c1c18e274d3db5"} Oct 30 18:36:56.992670 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:56.992629 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rpdb6" event={"ID":"f0e02e4a-2151-4339-93de-09057afd8f9d","Type":"ContainerStarted","Data":"9e3f7a064d7ecc66aceeb59df70b53476b769becc0e763c60440b08e54b19998"} Oct 30 18:36:56.995260 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:56.995224 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" event={"ID":"e4e82816-cfc3-413b-a254-47dd3a2535ea","Type":"ContainerStarted","Data":"f1289b64b7a5979a828e7aed105e6caaba36f5e11241de6d1a997e48684db0ce"} Oct 30 18:36:56.996990 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:56.996968 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" event={"ID":"7f5e5aef-db22-432b-9bfc-64231443a33c","Type":"ContainerStarted","Data":"f10ae32adc060e8152dd4f6b35f3af6315e96758532d912b2ae7548b75259c04"} Oct 30 18:36:57.112670 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.112620 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rpdb6" podStartSLOduration=4.543555919 podStartE2EDuration="13.11260875s" totalImagesPullingTime="8.569052831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:57.112172437 +0000 UTC m=+13.740017620" watchObservedRunningTime="2026-10-30 18:36:57.11260875 +0000 UTC m=+13.740453934" Oct 30 18:36:57.393169 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.393060 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" podStartSLOduration=3.940506412 podStartE2EDuration="13.393045838s" totalImagesPullingTime="9.452539426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:57.259907941 +0000 UTC m=+13.887753126" watchObservedRunningTime="2026-10-30 18:36:57.393045838 +0000 UTC m=+14.020891022" Oct 30 18:36:57.393333 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.393275 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" podStartSLOduration=3.079059119 podStartE2EDuration="13.393267264s" totalImagesPullingTime="10.314208145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:57.392857035 +0000 UTC m=+14.020702223" watchObservedRunningTime="2026-10-30 18:36:57.393267264 +0000 UTC m=+14.021112452" Oct 30 18:36:57.842476 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.842450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:57.842625 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:57.842582 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:36:57.842968 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.842946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:57.843035 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:57.843020 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:57.843108 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:57.843081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:57.843182 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:57.843168 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:58.001282 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:58.001250 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="761c5b78d193b1e4187f24081e487d147845f06ee75cfd06c203fa0e6d4afc39" exitCode=0 Oct 30 18:36:58.001820 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:58.001356 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"761c5b78d193b1e4187f24081e487d147845f06ee75cfd06c203fa0e6d4afc39"} Oct 30 18:36:58.498361 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:58.498327 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zllgp_7268393d-9be4-4e1c-87ff-c9ac7cfd7072/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:58.789920 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:58.789848 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c96l5_ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7/dns-node-resolver/0.log" Oct 30 18:36:58.791032 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:58.791006 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zllgp_7268393d-9be4-4e1c-87ff-c9ac7cfd7072/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:59.322572 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.322546 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:59.323073 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.322661 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:36:59.323418 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.323242 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:36:59.588647 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.588608 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxg89_05a02e6e-236c-4782-95af-7e99fa284781/node-ca/0.log" Oct 30 18:36:59.842834 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.842754 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:36:59.842834 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.842814 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:36:59.843009 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:36:59.842834 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:36:59.843009 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:59.842894 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:36:59.843009 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:59.842953 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:36:59.843116 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:36:59.843041 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:00.009117 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:00.009064 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="d70b742fa196217c551daf91a29659b5b976a683db309efabde5b9d5d48c8c4a" exitCode=0 Oct 30 18:37:00.009282 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:00.009135 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"d70b742fa196217c551daf91a29659b5b976a683db309efabde5b9d5d48c8c4a"} Oct 30 18:37:01.015437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015156 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"70b1b3e744cf7008bc7ec64d58697f18e6fde80d8eccc0e4349a6f4271230797"} Oct 30 18:37:01.015972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015455 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"c388ad8d6373c563e1d25bea55ce3d913fb7180494a10aee0dd2d92bd77ed62d"} Oct 30 18:37:01.015972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015516 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"7fdf61a3e0985f5634ff238f8a58f24047e0a29276ab29254c5da11aa79ead42"} Oct 30 18:37:01.015972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015564 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"23a463eadd2476b9d844af537d45f7a6c1b47e783fbf7e0e4d52f9a2c467a449"} Oct 30 18:37:01.015972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015610 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"9caf422d9082a72615baaa15b1846338b06154e73c09e755840e41ef07b65b08"} Oct 30 18:37:01.015972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.015654 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"a5ac681c1444bdbe0fb0bc807f15b2c183cf10ca4c064f9fc64f86f329240074"} Oct 30 18:37:01.621207 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.620981 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:01.621366 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.621254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:01.621366 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.621128 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:01.621366 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.621354 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:17.621333065 +0000 UTC m=+34.249178264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:01.621529 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.621419 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:01.621529 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.621467 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:17.621454379 +0000 UTC m=+34.249299564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:01.721898 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.721842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:01.722145 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.721996 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:01.722145 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.722015 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:01.722145 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.722027 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:01.722145 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.722111 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:17.722076741 +0000 UTC m=+34.349921918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:01.842078 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.842050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:01.842259 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.842053 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:01.842259 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:01.842053 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:01.842259 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.842217 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:01.842428 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.842313 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:01.842428 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:01.842405 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:03.020759 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.020727 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="bb8f9c196f093b3cae73499c1b3c439c3ceba373381ee113ccb450a62c65d05b" exitCode=0 Oct 30 18:37:03.021185 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.020810 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"bb8f9c196f093b3cae73499c1b3c439c3ceba373381ee113ccb450a62c65d05b"} Oct 30 18:37:03.024080 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.024045 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"4a0fcaf8cca4ef00ac7ccc5bb7b31289542b15d632cd538dbf215832c8e871a5"} Oct 30 18:37:03.843395 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.843366 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:03.843513 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:03.843444 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:03.843572 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.843553 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:03.843690 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:03.843678 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:03.843722 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:03.843680 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:03.843767 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:03.843753 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:04.028472 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:04.028444 2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="726f86ac5694fae4f3ded120c54093f54ea40e6864ed185661d99e51063717cc" exitCode=0 Oct 30 18:37:04.028894 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:04.028491 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerDied","Data":"726f86ac5694fae4f3ded120c54093f54ea40e6864ed185661d99e51063717cc"} Oct 30 18:37:05.034555 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.034524 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jnv56" event={"ID":"8bf1866b-1e63-4886-830d-c8645a8e5ff5","Type":"ContainerStarted","Data":"bee9b38b54434fda971a95b598712d2bf3df0a5133b76701423617a06b4eb751"} Oct 30 18:37:05.038618 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.038588 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" event={"ID":"52ca3b9e-8619-467a-a705-5bcbabdd7a08","Type":"ContainerStarted","Data":"5836b48e8185fe67bb9a6dc2f15a5ed4811699f40948493a52fa952611e3cccc"} Oct 30 18:37:05.038852 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.038835 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:05.038931 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.038906 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:05.039004 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.038955 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:05.056029 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.056007 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:05.056175 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.056162 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:05.183217 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.183019 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jnv56" podStartSLOduration=8.811955565 podStartE2EDuration="21.183005343s" totalImagesPullingTime="12.371049778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:05.182945376 +0000 UTC m=+21.810790559" watchObservedRunningTime="2026-10-30 18:37:05.183005343 +0000 UTC m=+21.810850526" Oct 30 18:37:05.331199 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.331079 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" podStartSLOduration=7.065751401 podStartE2EDuration="21.331064877s" totalImagesPullingTime="14.265313476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:05.331009961 +0000 UTC m=+21.958855147" watchObservedRunningTime="2026-10-30 18:37:05.331064877 +0000 UTC m=+21.958910470" Oct 30 18:37:05.842877 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.842845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:05.842877 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.842857 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:05.843076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:05.843008 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:05.843076 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:05.843008 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:05.843217 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:05.843190 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:05.843332 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:05.843308 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:07.842059 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:07.842027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:07.842516 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:07.842110 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:07.842516 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:07.842107 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:07.842516 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:07.842198 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:07.842516 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:07.842254 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:07.842516 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:07.842321 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:09.842185 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:09.842150 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:09.842185 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:09.842166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:09.842647 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:09.842249 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:09.842647 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:09.842320 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:09.842647 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:09.842345 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:09.842647 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:09.842428 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:11.842330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:11.842287 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:11.842330 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:11.842292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:11.842787 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:11.842436 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:11.842787 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:11.842481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:11.842787 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:11.842577 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:11.842787 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:11.842665 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:13.843108 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:13.843064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:13.843517 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:13.843145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:13.843517 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:13.843215 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:13.843517 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:13.843333 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:13.843517 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:13.843388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:13.843517 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:13.843465 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:15.842114 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:15.842054 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:15.842630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:15.842065 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:15.842630 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:15.842218 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:15.842630 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:15.842281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:15.842630 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:15.842394 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:15.842630 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:15.842504 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:17.635164 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.635127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:17.635734 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.635182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:17.635734 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.635273 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:17.635734 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.635278 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:17.635734 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.635327 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs podName:166168ae-19c0-4650-9a99-be2f63f2917f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:49.63531194 +0000 UTC m=+66.263157102 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs") pod "network-metrics-daemon-sndlp" (UID: "166168ae-19c0-4650-9a99-be2f63f2917f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:17.635734 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.635341 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret podName:182b13f1-6909-4251-8455-eed4d52ae204 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:49.63533482 +0000 UTC m=+66.263179981 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret") pod "global-pull-secret-syncer-plzrc" (UID: "182b13f1-6909-4251-8455-eed4d52ae204") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:17.736425 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.736397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:17.736583 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.736502 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:17.736583 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.736515 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:17.736583 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.736524 2579 projected.go:196] Error preparing data for projected volume kube-api-access-9h9mm for pod openshift-network-diagnostics/network-check-target-wnj4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:17.736583 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.736565 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm podName:7190032a-e51d-4bd0-95e5-30f6f95e59a4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:49.73655269 +0000 UTC m=+66.364397852 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9h9mm" (UniqueName: "kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm") pod "network-check-target-wnj4b" (UID: "7190032a-e51d-4bd0-95e5-30f6f95e59a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:17.844948 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.844915 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:17.845078 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.845020 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:17.845078 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.845076 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:17.845232 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.845192 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:17.845352 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:17.845236 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:17.845352 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:17.845337 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:19.643053 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.643021 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnj4b"] Oct 30 18:37:19.643530 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.643183 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:19.643530 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:19.643266 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:19.646620 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.646596 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-plzrc"] Oct 30 18:37:19.646739 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.646693 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:19.646790 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:19.646774 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:19.647072 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.647054 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sndlp"] Oct 30 18:37:19.647172 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:19.647161 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:19.647250 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:19.647236 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:20.842166 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:20.842140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:20.842504 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:20.842169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:20.842504 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:20.842234 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:20.842504 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:20.842353 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:21.842228 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:21.842192 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:21.842766 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:21.842295 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:22.842974 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:22.842943 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:22.843342 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:22.843045 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:22.843342 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:22.843133 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:22.843342 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:22.843223 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:23.844697 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:23.844665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:23.845116 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:23.844768 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnj4b" podUID="7190032a-e51d-4bd0-95e5-30f6f95e59a4" Oct 30 18:37:24.842440 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:24.842286 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:24.842550 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:24.842336 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:24.842608 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:24.842538 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sndlp" podUID="166168ae-19c0-4650-9a99-be2f63f2917f" Oct 30 18:37:24.842608 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:24.842578 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-plzrc" podUID="182b13f1-6909-4251-8455-eed4d52ae204" Oct 30 18:37:25.156869 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.156806 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeReady" Oct 30 18:37:25.157227 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.156901 2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:37:25.269158 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.269132 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7txjh"] Oct 30 18:37:25.270977 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.270962 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.272131 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.272111 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j22f2"] Oct 30 18:37:25.273583 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.273569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.326463 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.326439 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:37:25.327991 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.327965 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k96h\"" Oct 30 18:37:25.328113 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.328081 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:37:25.328964 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.328948 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:37:25.329957 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.329902 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7txjh"] Oct 30 18:37:25.332807 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.332790 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j22f2"] Oct 30 18:37:25.369701 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.369682 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rd622"] Oct 30 18:37:25.371301 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.371289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.396358 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6cc6\" (UniqueName: \"kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.396430 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.396430 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfk42\" (UniqueName: \"kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.396430 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.396543 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.396580 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.396552 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.426596 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.426547 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:37:25.427628 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.427606 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:37:25.428437 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.428421 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:37:25.428510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.428477 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-kxkbv\"" Oct 30 18:37:25.428510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.428496 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:37:25.428816 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.428800 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5fxhm\"" Oct 30 18:37:25.428889 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.428876 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rd622"] Oct 30 18:37:25.497708 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497689 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.497759 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6sh\" (UniqueName: \"kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.497759 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.497858 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.497932 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.497982 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.497982 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.497957 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6cc6\" (UniqueName: \"kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.498068 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.498068 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfk42\" (UniqueName: \"kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.498181 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.498181 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.498347 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498330 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.498533 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.498516 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.501886 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.501860 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.501963 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.501901 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.527608 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.527585 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:37:25.527683 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.527649 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:37:25.559631 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.559604 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6cc6\" (UniqueName: \"kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.559721 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.559644 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfk42\" (UniqueName: \"kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.583261 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.583245 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:25.587427 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.587412 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:25.599311 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599410 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599410 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599528 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599528 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599447 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6sh\" (UniqueName: \"kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599661 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599636 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.599743 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.599716 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.600278 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.600258 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.601679 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.601662 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.660750 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.660678 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6sh\" (UniqueName: \"kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.681489 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.681440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:25.842550 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.842521 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:25.898387 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.898361 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:37:25.898387 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.898374 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:37:25.898710 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.898697 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mrt5r\"" Oct 30 18:37:25.965946 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:25.965922 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70452c4_c454_4edd_857c_3eb8d36acd12.slice/crio-2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c WatchSource:0}: Error finding container 2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c: Status 404 returned error can't find the container with id 2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c Oct 30 18:37:25.966870 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:25.966840 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3740ec35_b3ca_4b78_a3f2_610eeb7b3ada.slice/crio-9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff WatchSource:0}: Error finding container 9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff: Status 404 returned error can't find the container with id 9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff Oct 30 18:37:25.967366 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.967347 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j22f2"] Oct 30 18:37:25.969425 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:25.969406 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7txjh"] Oct 30 18:37:26.043737 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:26.043717 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab46616b_1670_40b4_97be_689b9931022c.slice/crio-6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870 WatchSource:0}: Error finding container 6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870: Status 404 returned error can't find the container with id 6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870 Oct 30 18:37:26.046070 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.046051 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rd622"] Oct 30 18:37:26.081638 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.081510 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd622" event={"ID":"ab46616b-1670-40b4-97be-689b9931022c","Type":"ContainerStarted","Data":"6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870"} Oct 30 18:37:26.082847 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.082777 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j22f2" event={"ID":"e70452c4-c454-4edd-857c-3eb8d36acd12","Type":"ContainerStarted","Data":"2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c"} Oct 30 18:37:26.083996 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.083973 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7txjh" event={"ID":"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada","Type":"ContainerStarted","Data":"9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff"} Oct 30 18:37:26.842909 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.842880 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:26.842909 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.842906 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:26.898871 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.898842 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hd9r2\"" Oct 30 18:37:26.898979 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.898905 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kp722\"" Oct 30 18:37:26.898979 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.898910 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:37:26.898979 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:26.898914 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:37:27.089077 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:27.089033 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd622" event={"ID":"ab46616b-1670-40b4-97be-689b9931022c","Type":"ContainerStarted","Data":"b6bbdc148849145b0337ab07ca9817a4bc72ab1c5b08fdc2f570503dc011cc3c"} Oct 30 18:37:28.094623 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:28.094576 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd622" event={"ID":"ab46616b-1670-40b4-97be-689b9931022c","Type":"ContainerStarted","Data":"985665e8b2212637baca96413f5d8549f45658918b4d09f7313530d3988fe542"} Oct 30 18:37:29.099519 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:29.099481 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7txjh" event={"ID":"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada","Type":"ContainerStarted","Data":"60e4f26a6a163208b04f02ccea0fd4fb93c945b8a18976be24224b7ad032f4f9"} Oct 30 18:37:29.099989 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:29.099540 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7txjh" event={"ID":"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada","Type":"ContainerStarted","Data":"5506eca53681f077f45b455d54f1c8586078c7698ab6ae663b81fcf368bbcdb4"} Oct 30 18:37:29.100361 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:29.100331 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:29.222697 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:29.222647 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-7txjh" podStartSLOduration=2.034002715 podStartE2EDuration="4.222629746s" totalImagesPullingTime="2.188627031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:29.221676371 +0000 UTC m=+45.849521556" watchObservedRunningTime="2026-10-30 18:37:29.222629746 +0000 UTC m=+45.850474933" Oct 30 18:37:30.105023 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.104256 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j22f2" event={"ID":"e70452c4-c454-4edd-857c-3eb8d36acd12","Type":"ContainerStarted","Data":"55fab60a343234d2c4340335803bb10d9425f599040afab2ac346854573f59d2"} Oct 30 18:37:30.107226 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.107194 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rd622" event={"ID":"ab46616b-1670-40b4-97be-689b9931022c","Type":"ContainerStarted","Data":"89a919d029fa5951c357903ecd7b7a407480c970a0cba4ff6f0753869950e3b9"} Oct 30 18:37:30.225431 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.225390 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j22f2" podStartSLOduration=1.977166227 podStartE2EDuration="5.22537918s" totalImagesPullingTime="3.248212953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:30.225323279 +0000 UTC m=+46.853168458" watchObservedRunningTime="2026-10-30 18:37:30.22537918 +0000 UTC m=+46.853224364" Oct 30 18:37:30.357340 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.357262 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rd622" podStartSLOduration=2.2663799940000002 podStartE2EDuration="5.357248641s" totalImagesPullingTime="3.090868647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:30.356742403 +0000 UTC m=+46.984587589" watchObservedRunningTime="2026-10-30 18:37:30.357248641 +0000 UTC m=+46.985093835" Oct 30 18:37:30.621497 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.621434 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm"] Oct 30 18:37:30.674455 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.674429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.682955 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.682929 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm"] Oct 30 18:37:30.732077 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.732056 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:37:30.733058 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.733013 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:37:30.733195 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.733180 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:37:30.733275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.733263 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-h6fl7\"" Oct 30 18:37:30.733336 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.733331 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:37:30.737640 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.737619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qpzt\" (UniqueName: \"kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.737724 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.737658 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.737785 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.737771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.737821 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.737805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.782076 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.782056 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 18:37:30.838133 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.838111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.838231 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.838164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.838231 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.838188 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.838231 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.838213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpzt\" (UniqueName: \"kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.838881 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.838861 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.842080 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.842062 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.842192 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.842061 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.862589 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.862569 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv"] Oct 30 18:37:30.883800 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.883753 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w8v99"] Oct 30 18:37:30.883893 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.883875 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.900084 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.900064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.901738 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.901703 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpzt\" (UniqueName: \"kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:30.932156 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.932134 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv"] Oct 30 18:37:30.939392 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939398 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939417 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939483 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939432 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939585 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939627 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939656 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939686 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939717 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v562v\" (UniqueName: \"kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939751 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939716 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939751 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sg2c\" (UniqueName: \"kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939812 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939846 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:30.939929 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.939929 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.939866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:30.943164 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.943143 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:37:30.943247 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.943187 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 18:37:30.943247 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.943196 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q9k2d\"" Oct 30 18:37:30.943765 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:30.943751 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 18:37:31.026922 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.026900 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:31.040275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040314 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040502 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040502 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040435 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040502 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040610 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040610 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040538 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040610 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v562v\" (UniqueName: \"kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040610 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040610 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040596 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sg2c\" (UniqueName: \"kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.040855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.040855 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040796 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.041069 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040862 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.041069 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040926 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.041069 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.040931 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.041368 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.041345 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.041404 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.041369 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.041436 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.041424 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.043282 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043260 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:37:31.043367 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043315 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6qrp7\"" Oct 30 18:37:31.043421 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043371 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.043421 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043384 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:37:31.043521 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043474 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.043581 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.043553 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:37:31.051420 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.051393 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.053179 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.053157 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.053252 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.053217 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.106077 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.106045 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v562v\" (UniqueName: \"kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.120187 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.120164 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sg2c\" (UniqueName: \"kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.235172 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.235142 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:31.278662 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.278628 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:31.308678 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:31.308652 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1df3f31_27af_4f32_aa1e_e07cf031f2cf.slice/crio-9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c WatchSource:0}: Error finding container 9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c: Status 404 returned error can't find the container with id 9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c Oct 30 18:37:31.470384 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:31.470360 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6523be_9435_46c7_b5a2_b84a2bba0d83.slice/crio-f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe WatchSource:0}: Error finding container f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe: Status 404 returned error can't find the container with id f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe Oct 30 18:37:31.472967 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.472948 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm"] Oct 30 18:37:31.616850 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:31.616829 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e792e9_28c1_4127_b364_3679fcf27c7c.slice/crio-2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a WatchSource:0}: Error finding container 2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a: Status 404 returned error can't find the container with id 2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a Oct 30 18:37:31.618808 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:31.618787 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv"] Oct 30 18:37:32.113415 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:32.113383 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8v99" event={"ID":"a1df3f31-27af-4f32-aa1e-e07cf031f2cf","Type":"ContainerStarted","Data":"9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c"} Oct 30 18:37:32.115214 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:32.115184 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" event={"ID":"7a6523be-9435-46c7-b5a2-b84a2bba0d83","Type":"ContainerStarted","Data":"45b7d63ed5b4f9f5cf24923359b6337a5ade32f6624c128fc0940eeca77c300f"} Oct 30 18:37:32.115320 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:32.115232 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" event={"ID":"7a6523be-9435-46c7-b5a2-b84a2bba0d83","Type":"ContainerStarted","Data":"b092f15860fcdee87812e420ac0bed975d6fd3db4beb0a2328a20e7aee181fd6"} Oct 30 18:37:32.115320 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:32.115259 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" event={"ID":"7a6523be-9435-46c7-b5a2-b84a2bba0d83","Type":"ContainerStarted","Data":"f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe"} Oct 30 18:37:32.116210 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:32.116190 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" event={"ID":"79e792e9-28c1-4127-b364-3679fcf27c7c","Type":"ContainerStarted","Data":"2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a"} Oct 30 18:37:33.019966 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.019929 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:37:33.037415 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.037383 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.052873 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.052847 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053008 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.052884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053008 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.052919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053008 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.052997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053185 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053185 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053269 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053306 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053306 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053390 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053320 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053390 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053473 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.053528 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.053469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.092951 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.092926 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:37:33.092951 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.092942 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:37:33.093297 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.093280 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:37:33.093420 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.093403 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lr79t\"" Oct 30 18:37:33.144870 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.144845 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:37:33.144870 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.144845 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:37:33.145507 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.145489 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:37:33.145632 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.145540 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:37:33.145632 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.145589 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:37:33.146225 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.146206 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:37:33.154023 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154115 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154115 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154115 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154275 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154310 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154423 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154456 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154510 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.154784 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.154701 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.157438 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.157384 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.157927 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.157897 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.157927 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.157921 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.158714 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.158410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.158714 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.158530 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.158714 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.158618 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.158714 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.158675 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.159200 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.159177 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.159370 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.159352 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.202327 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.202300 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:37:33.204751 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.204731 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.225830 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.225769 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.412886 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.412859 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c8658f985-v8kgg"] Oct 30 18:37:33.425062 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.425036 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.434209 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.434191 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:33.455529 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455504 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455594 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455554 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455642 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455710 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455746 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455746 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455825 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455800 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455900 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.455964 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.455924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjk5\" (UniqueName: \"kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.483483 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c8658f985-v8kgg"] Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484082 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484276 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484314 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r7d57\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484390 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484675 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484726 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 18:37:33.485323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.484760 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-35h58a6i46adv\"" Oct 30 18:37:33.557325 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557209 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557325 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557325 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557454 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjk5\" (UniqueName: \"kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.557569 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.557513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.558909 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.558589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.560902 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.560878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.560972 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.560931 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.561233 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.561214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.561838 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.561806 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.566922 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.566901 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.567002 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.566899 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.567002 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.566974 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.620504 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.620482 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjk5\" (UniqueName: \"kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.866082 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.866014 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:33.909846 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:33.909814 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cb5f50_428f_433b_b372_5b6e19584741.slice/crio-d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137 WatchSource:0}: Error finding container d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137: Status 404 returned error can't find the container with id d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137 Oct 30 18:37:33.912526 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:33.912501 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:37:34.124852 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.124795 2579 generic.go:356] "Generic (PLEG): container finished" podID="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" containerID="4016e43963013a87dd0665678fa622eb9b8ebd0aa14485c0a277d3b987555170" exitCode=0 Oct 30 18:37:34.124970 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.124864 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8v99" event={"ID":"a1df3f31-27af-4f32-aa1e-e07cf031f2cf","Type":"ContainerDied","Data":"4016e43963013a87dd0665678fa622eb9b8ebd0aa14485c0a277d3b987555170"} Oct 30 18:37:34.126219 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.126197 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137"} Oct 30 18:37:34.128049 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.128028 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" event={"ID":"7a6523be-9435-46c7-b5a2-b84a2bba0d83","Type":"ContainerStarted","Data":"0e33c26e2bbd0da015ff780790423d4d55f0cede8ef5b2977fb1d5b1c0ec8cbc"} Oct 30 18:37:34.254765 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:34.254743 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdde0beb_f496_4013_aff3_3706e199e7f2.slice/crio-211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05 WatchSource:0}: Error finding container 211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05: Status 404 returned error can't find the container with id 211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05 Oct 30 18:37:34.256372 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.256352 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c8658f985-v8kgg"] Oct 30 18:37:34.373755 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:34.373702 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" podStartSLOduration=2.543794196 podStartE2EDuration="4.373682444s" totalImagesPullingTime="1.829888248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:34.37287199 +0000 UTC m=+51.000717175" watchObservedRunningTime="2026-10-30 18:37:34.373682444 +0000 UTC m=+51.001527628" Oct 30 18:37:35.132900 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.132865 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" event={"ID":"79e792e9-28c1-4127-b364-3679fcf27c7c","Type":"ContainerStarted","Data":"1c181c5d98ac66545c940f912d3bd90f1e9e085f5efc8864e539ca811404533c"} Oct 30 18:37:35.133068 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.132936 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" event={"ID":"79e792e9-28c1-4127-b364-3679fcf27c7c","Type":"ContainerStarted","Data":"f9b2e58db491fad333c087ab9876ba3f890ae19b8aee5ca24ac9081eb02aa6c1"} Oct 30 18:37:35.133068 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.132964 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" event={"ID":"79e792e9-28c1-4127-b364-3679fcf27c7c","Type":"ContainerStarted","Data":"8a0479e65499dce33a3ea8007b4b833a11f9c13922d7b752842845b59a526076"} Oct 30 18:37:35.134973 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.134941 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8v99" event={"ID":"a1df3f31-27af-4f32-aa1e-e07cf031f2cf","Type":"ContainerStarted","Data":"f06b61ec7cae5fcc1f2515fb9cd71055ff1f5cf78c65c8e56cb0ff30b2fff577"} Oct 30 18:37:35.135085 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.134997 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8v99" event={"ID":"a1df3f31-27af-4f32-aa1e-e07cf031f2cf","Type":"ContainerStarted","Data":"5ae83d07a2573ed8f8e800184cef671dc844c8222f1c34b4413d122515999506"} Oct 30 18:37:35.136057 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.136034 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05"} Oct 30 18:37:35.263162 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.263077 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" podStartSLOduration=2.635537919 podStartE2EDuration="5.263060429s" totalImagesPullingTime="2.62752251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:35.262358015 +0000 UTC m=+51.890203198" watchObservedRunningTime="2026-10-30 18:37:35.263060429 +0000 UTC m=+51.890905612" Oct 30 18:37:35.395478 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.395376 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w8v99" podStartSLOduration=3.3515375609999998 podStartE2EDuration="5.39535524s" totalImagesPullingTime="1.961932986s" totalInitContainerRuntime="81.884693ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:35.39441286 +0000 UTC m=+52.022258044" watchObservedRunningTime="2026-10-30 18:37:35.39535524 +0000 UTC m=+52.023200425" Oct 30 18:37:35.487863 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.487831 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2"] Oct 30 18:37:35.511499 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.511476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.545132 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.544958 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2"] Oct 30 18:37:35.573766 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.573739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.573886 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.573777 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g84q\" (UniqueName: \"kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.585228 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.585206 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 18:37:35.585335 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.585289 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9rjdp\"" Oct 30 18:37:35.674964 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.674939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g84q\" (UniqueName: \"kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.675120 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.675083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.677812 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.677786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.742344 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.742317 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g84q\" (UniqueName: \"kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.771937 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.771915 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56686866c6-xhcgv"] Oct 30 18:37:35.793649 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.793624 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.832697 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.832662 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56686866c6-xhcgv"] Oct 30 18:37:35.835593 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.835574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:35.849272 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.849246 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 18:37:35.849383 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.849358 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 18:37:35.849464 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.849445 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 18:37:35.852623 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.852598 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5l64w\"" Oct 30 18:37:35.852737 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.852715 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cj5hm9jbahbsc\"" Oct 30 18:37:35.852788 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.852776 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 18:37:35.876507 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876484 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876592 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876533 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876592 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876709 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd2dc\" (UniqueName: \"kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876709 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876642 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876709 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.876813 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.876723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.977949 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.977918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978056 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.977975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978056 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.977996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978056 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2dc\" (UniqueName: \"kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978056 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978576 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978576 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978576 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978530 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.978775 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.978741 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.979323 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.979262 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.980596 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.980576 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.980993 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.980977 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:35.981183 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:35.981163 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:36.038902 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.038883 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2dc\" (UniqueName: \"kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:36.106084 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.106064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:36.140316 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.140292 2579 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" exitCode=0 Oct 30 18:37:36.140417 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.140363 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} Oct 30 18:37:36.234357 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:36.234328 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef44a6cf_b792_4c9c_8503_271d365976f8.slice/crio-772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2 WatchSource:0}: Error finding container 772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2: Status 404 returned error can't find the container with id 772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2 Oct 30 18:37:36.236625 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.236599 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2"] Oct 30 18:37:36.502009 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:36.501959 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734c247a_5ffb_4ccc_9720_8ba9d5ac3711.slice/crio-7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186 WatchSource:0}: Error finding container 7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186: Status 404 returned error can't find the container with id 7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186 Oct 30 18:37:36.504595 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:36.504573 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56686866c6-xhcgv"] Oct 30 18:37:37.057591 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:37.057566 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:37.145784 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:37.145759 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" event={"ID":"ef44a6cf-b792-4c9c-8503-271d365976f8","Type":"ContainerStarted","Data":"772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2"} Oct 30 18:37:37.146806 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:37.146783 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" event={"ID":"734c247a-5ffb-4ccc-9720-8ba9d5ac3711","Type":"ContainerStarted","Data":"7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186"} Oct 30 18:37:38.161784 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:38.161744 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"926fbc6c2431633600346a79f6e473c853b7028ca07acc056d5d1ff789c868b8"} Oct 30 18:37:38.162551 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:38.162528 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"0d672ac4b1d04ff21724989398e758e0aaa0eba35a84e32f1ab6c29b6fe505d6"} Oct 30 18:37:38.162718 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:38.162703 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"a7d9c32ca373f8f37a81181623459a27739e98d256430f54f0a257e38a35ae1a"} Oct 30 18:37:39.168880 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.168842 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} Oct 30 18:37:39.169398 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.168923 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} Oct 30 18:37:39.169398 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.168966 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} Oct 30 18:37:39.169398 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.169011 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} Oct 30 18:37:39.169398 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.169051 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} Oct 30 18:37:39.169398 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.169118 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerStarted","Data":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} Oct 30 18:37:39.172128 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.172079 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"8f11d825384f46ceab9715c76396108a4b8c15140b017000e5d0b33550895477"} Oct 30 18:37:39.172271 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.172168 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"de8cc7ecc113b160e65da1ab79a4d76fa670dd6e23544b5ff2c8f44a0c54dd3d"} Oct 30 18:37:39.172271 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.172209 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" event={"ID":"bdde0beb-f496-4013-aff3-3706e199e7f2","Type":"ContainerStarted","Data":"a2f36290b5e491311b8c7db172806483bb9b93c59877ee09e0e464ee76e7057b"} Oct 30 18:37:39.172271 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.172251 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:39.173682 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.173659 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" event={"ID":"ef44a6cf-b792-4c9c-8503-271d365976f8","Type":"ContainerStarted","Data":"317fe3c474a78698b13a77b1ad46e40b674c9e54b2278b84a33a7734609f2ce2"} Oct 30 18:37:39.173817 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.173799 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:39.175337 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.175309 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" event={"ID":"734c247a-5ffb-4ccc-9720-8ba9d5ac3711","Type":"ContainerStarted","Data":"3ce5f72778dcbbdd014c6d47a0c82f96779f91bdf2e24f11d0800f63503338a8"} Oct 30 18:37:39.178993 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.178977 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:39.354723 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.354681 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.189790711 podStartE2EDuration="7.354670724s" totalImagesPullingTime="4.141316297s" totalInitContainerRuntime="23.563716ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:39.353338333 +0000 UTC m=+55.981183518" watchObservedRunningTime="2026-10-30 18:37:39.354670724 +0000 UTC m=+55.982515908" Oct 30 18:37:39.488341 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.488294 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" podStartSLOduration=2.335531401 podStartE2EDuration="6.488279606s" totalImagesPullingTime="4.152748205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:39.487763736 +0000 UTC m=+56.115608917" watchObservedRunningTime="2026-10-30 18:37:39.488279606 +0000 UTC m=+56.116124790" Oct 30 18:37:39.606764 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.606709 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" podStartSLOduration=2.4927770479999998 podStartE2EDuration="4.606695435s" totalImagesPullingTime="2.113918387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:39.606584316 +0000 UTC m=+56.234429497" watchObservedRunningTime="2026-10-30 18:37:39.606695435 +0000 UTC m=+56.234540621" Oct 30 18:37:39.736821 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:39.736775 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" podStartSLOduration=2.389780666 podStartE2EDuration="4.736761095s" totalImagesPullingTime="2.346980429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:39.735259741 +0000 UTC m=+56.363104925" watchObservedRunningTime="2026-10-30 18:37:39.736761095 +0000 UTC m=+56.364606289" Oct 30 18:37:40.113854 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.113823 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:40.189520 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.189492 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:37:40.193397 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.193379 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218066 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218234 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218528 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218631 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218698 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218874 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218992 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.218992 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.218941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219080 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219179 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219236 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219313 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219296 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219438 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219420 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219588 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219567 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219684 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219718 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219892 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219933 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219906 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.219933 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.219928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.220058 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.220040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.267881 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.267862 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:37:40.268065 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268052 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:37:40.268143 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268067 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:37:40.268143 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268129 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:37:40.268217 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268190 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tj5qj\"" Oct 30 18:37:40.268290 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268275 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:37:40.268328 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268304 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:37:40.268328 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268315 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:37:40.268397 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268311 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:37:40.268397 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268330 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:37:40.268462 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268412 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:37:40.268499 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268476 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:37:40.268553 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268540 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:37:40.268589 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.268543 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-acesbpb1nq1nl\"" Oct 30 18:37:40.320463 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320587 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320587 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320491 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320587 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320587 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.320803 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320820 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320940 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.320978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.321001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321159 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.321032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321533 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.321440 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.321785 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.321690 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.323113 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.323073 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:37:40.323281 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.323257 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.323596 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.323574 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:37:40.324451 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.324423 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.324653 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.324605 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.324721 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.324705 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.325600 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.325429 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.325903 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.325876 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.327826 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.327803 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.327952 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.327803 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328011 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.327958 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328139 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.328124 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328204 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.328148 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328332 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.328309 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328384 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.328366 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.328523 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.328503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.329346 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.329325 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.330079 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.330056 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.334044 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.334022 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.358608 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.358580 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:37:40.382969 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.382925 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.518662 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.518635 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:40.950351 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:40.950317 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd193331c_eb89_4849_b124_76b1bf0d0263.slice/crio-a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb WatchSource:0}: Error finding container a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb: Status 404 returned error can't find the container with id a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb Oct 30 18:37:40.951706 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:40.951678 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:37:41.183358 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:41.183326 2579 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" exitCode=0 Oct 30 18:37:41.183486 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:41.183424 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} Oct 30 18:37:41.183560 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:41.183543 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb"} Oct 30 18:37:45.184088 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.184065 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:45.203866 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.203836 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} Oct 30 18:37:45.203966 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.203904 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} Oct 30 18:37:45.203966 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.203943 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} Oct 30 18:37:45.204035 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.203983 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} Oct 30 18:37:45.204035 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.204023 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} Oct 30 18:37:45.204131 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.204062 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerStarted","Data":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} Oct 30 18:37:45.457114 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.457048 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.865503425 podStartE2EDuration="5.457033721s" totalImagesPullingTime="3.576802934s" totalInitContainerRuntime="14.727362ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:45.454348229 +0000 UTC m=+62.082193414" watchObservedRunningTime="2026-10-30 18:37:45.457033721 +0000 UTC m=+62.084878939" Oct 30 18:37:45.519029 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:45.518999 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:49.706612 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.706575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:49.707004 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.706656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:49.759662 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.759634 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:37:49.770203 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.770181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:49.807772 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.807746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:49.811291 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.811274 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:37:49.819498 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.819467 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:49.860505 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.860483 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:37:49.920954 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.920931 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:37:49.931487 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:49.931461 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:50.012241 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.012165 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kp722\"" Oct 30 18:37:50.012338 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.012313 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hd9r2\"" Oct 30 18:37:50.019333 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.019313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:50.019333 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.019327 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:50.207910 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.207880 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mrt5r\"" Oct 30 18:37:50.215208 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.215186 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:50.451539 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:50.451514 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166168ae_19c0_4650_9a99_be2f63f2917f.slice/crio-11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a WatchSource:0}: Error finding container 11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a: Status 404 returned error can't find the container with id 11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a Oct 30 18:37:50.453726 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.453707 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sndlp"] Oct 30 18:37:50.469938 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.469914 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-plzrc"] Oct 30 18:37:50.479322 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:50.479300 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182b13f1_6909_4251_8455_eed4d52ae204.slice/crio-2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb WatchSource:0}: Error finding container 2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb: Status 404 returned error can't find the container with id 2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb Oct 30 18:37:50.607293 ip-10-0-128-38 kubenswrapper[2579]: W1030 18:37:50.607270 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7190032a_e51d_4bd0_95e5_30f6f95e59a4.slice/crio-89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230 WatchSource:0}: Error finding container 89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230: Status 404 returned error can't find the container with id 89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230 Oct 30 18:37:50.610019 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:50.610001 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnj4b"] Oct 30 18:37:51.230180 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:51.230151 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-plzrc" event={"ID":"182b13f1-6909-4251-8455-eed4d52ae204","Type":"ContainerStarted","Data":"2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb"} Oct 30 18:37:51.231446 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:51.231421 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sndlp" event={"ID":"166168ae-19c0-4650-9a99-be2f63f2917f","Type":"ContainerStarted","Data":"11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a"} Oct 30 18:37:51.232540 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:51.232519 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnj4b" event={"ID":"7190032a-e51d-4bd0-95e5-30f6f95e59a4","Type":"ContainerStarted","Data":"89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230"} Oct 30 18:37:52.241627 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:52.240922 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sndlp" event={"ID":"166168ae-19c0-4650-9a99-be2f63f2917f","Type":"ContainerStarted","Data":"c4b4b7df9edabc05854a4c81a25acbc765484946b01df886803d45b6e5172d03"} Oct 30 18:37:52.241627 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:52.240972 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sndlp" event={"ID":"166168ae-19c0-4650-9a99-be2f63f2917f","Type":"ContainerStarted","Data":"262b920b215626ff790c851b676735e39a397c2047587c7c07562f75d9e32ffc"} Oct 30 18:37:52.378629 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:52.378569 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sndlp" podStartSLOduration=66.848759696 podStartE2EDuration="1m8.378550049s" totalImagesPullingTime="1.529790353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:52.377884948 +0000 UTC m=+69.005730137" watchObservedRunningTime="2026-10-30 18:37:52.378550049 +0000 UTC m=+69.006395234" Oct 30 18:37:55.254450 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:55.254413 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnj4b" event={"ID":"7190032a-e51d-4bd0-95e5-30f6f95e59a4","Type":"ContainerStarted","Data":"312dc0a78171f7af31ca7ca4e291525de5d518482426223346cdd47419507c80"} Oct 30 18:37:55.254959 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:55.254644 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:55.376491 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:55.376434 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wnj4b" podStartSLOduration=67.647010738 podStartE2EDuration="1m11.376417782s" totalImagesPullingTime="3.729407044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:55.375646592 +0000 UTC m=+72.003491776" watchObservedRunningTime="2026-10-30 18:37:55.376417782 +0000 UTC m=+72.004262967" Oct 30 18:37:55.711557 ip-10-0-128-38 kubenswrapper[2579]: E1030 18:37:55.711529 2579 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:37:55.874411 ip-10-0-128-38 kubenswrapper[2579]: I1030 18:37:55.874298 2579 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:55.874500 ip-10-0-128-38 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:37:55.883472 ip-10-0-128-38 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:37:55.883686 ip-10-0-128-38 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:37:55.883884 ip-10-0-128-38 systemd[1]: kubelet.service: Consumed 4.308s CPU time. Oct 30 18:37:55.895645 ip-10-0-128-38 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:37:55.959385 ip-10-0-128-38 kubenswrapper[5869]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:55.959385 ip-10-0-128-38 kubenswrapper[5869]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:37:55.959385 ip-10-0-128-38 kubenswrapper[5869]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:55.959385 ip-10-0-128-38 kubenswrapper[5869]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:55.961521 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961447 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.961521 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961519 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.961521 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961523 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.961521 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961526 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961529 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961532 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961535 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961538 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961541 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961543 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961546 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961549 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961552 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961554 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961557 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961559 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961562 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961565 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961567 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961570 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961580 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961583 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961586 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.961688 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961589 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961591 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961594 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961597 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961599 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961602 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961604 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961608 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961611 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961613 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961616 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961619 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961621 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961624 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961626 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961629 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961632 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961635 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961638 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.962218 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961640 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961643 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961645 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961649 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961653 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961657 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961660 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961662 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961665 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961668 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961670 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961673 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961675 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961679 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961682 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961684 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961687 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961691 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961694 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.962696 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961697 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961701 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961704 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961706 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961709 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961711 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961714 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961717 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961719 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961722 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961724 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961728 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961732 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961736 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961739 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961742 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961745 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961748 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961751 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.963229 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961754 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961756 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961759 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961761 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961764 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961767 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961769 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961772 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961775 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961777 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961780 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961783 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961786 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961788 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961791 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961793 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961796 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961798 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961801 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.963700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961803 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961806 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961809 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961811 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961814 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961817 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.961820 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962574 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962581 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962584 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962587 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962590 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962593 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962595 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962598 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962601 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962604 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962606 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962609 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962612 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.964190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962615 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962617 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962620 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962623 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962625 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962628 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962630 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962633 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962635 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962638 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962641 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962643 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962646 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962648 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962651 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962653 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962656 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962659 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962661 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962670 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.964732 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962672 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962675 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962678 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962680 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962683 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962685 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962688 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962691 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962693 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962696 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962700 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962703 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962705 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962708 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962711 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962713 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962716 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962718 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962721 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.965255 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962723 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962726 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962728 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962731 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962733 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962736 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962738 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962741 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962743 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962746 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962748 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962751 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962753 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962756 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962759 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962762 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962764 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962767 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962770 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.965724 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962775 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962779 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962782 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962785 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962790 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962793 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962795 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962798 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962801 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962803 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962806 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962809 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962811 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962814 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962817 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962819 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962822 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962826 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962829 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.966253 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962832 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962834 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962839 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962842 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962844 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962847 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962849 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962852 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962855 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962857 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962861 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962864 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962866 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962869 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962871 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.962874 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963376 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963385 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963388 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.966717 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963390 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963393 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963396 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963398 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963401 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963404 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963406 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963409 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963411 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963414 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963417 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963420 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963422 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963425 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963427 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963431 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963434 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963436 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963439 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.967225 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963441 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963444 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963447 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963450 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963452 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963455 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963458 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963461 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963463 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963466 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963468 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963473 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963476 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963481 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963484 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963487 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963489 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963492 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963495 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963498 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.967683 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963500 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963503 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963505 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963508 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963511 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963513 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963516 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963518 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963522 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963524 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963527 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963530 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963532 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963535 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963537 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963540 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963543 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963545 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963548 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.968190 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963551 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963553 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963555 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963558 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963562 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963564 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963567 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963569 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963572 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963574 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963577 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963579 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963582 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963584 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963587 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963590 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963594 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963598 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963601 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.968653 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963604 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963607 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963612 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963614 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963617 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963620 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963623 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963625 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963628 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963630 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963633 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963636 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963639 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963641 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963644 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963648 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963651 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963654 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963657 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963659 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.969149 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963662 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963665 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963668 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963670 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963673 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.963675 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963773 5869 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963784 5869 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963793 5869 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963797 5869 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963802 5869 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963805 5869 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963812 5869 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963816 5869 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963820 5869 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963823 5869 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963826 5869 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963830 5869 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963833 5869 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963837 5869 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963840 5869 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963842 5869 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963845 5869 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963849 5869 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:37:55.969639 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963853 5869 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963855 5869 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963859 5869 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963862 5869 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963865 5869 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963869 5869 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963872 5869 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963875 5869 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963878 5869 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963881 5869 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963884 5869 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963887 5869 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963890 5869 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963893 5869 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963897 5869 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963900 5869 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963903 5869 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963906 5869 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963909 5869 flags.go:64] FLAG: --enable-server="true" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963912 5869 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963916 5869 flags.go:64] FLAG: --event-burst="100" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963919 5869 flags.go:64] FLAG: --event-qps="50" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963923 5869 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963926 5869 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:37:55.970270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963929 5869 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963933 5869 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963936 5869 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963939 5869 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963942 5869 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963945 5869 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963948 5869 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963951 5869 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963954 5869 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963957 5869 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963960 5869 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963963 5869 flags.go:64] FLAG: --feature-gates="" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963968 5869 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963971 5869 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963974 5869 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963977 5869 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963980 5869 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963983 5869 flags.go:64] FLAG: --help="false" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963986 5869 flags.go:64] FLAG: --hostname-override="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963989 5869 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963992 5869 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963995 5869 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.963999 5869 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:37:55.971306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964003 5869 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964006 5869 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964009 5869 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964011 5869 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964014 5869 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964017 5869 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964020 5869 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964024 5869 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964027 5869 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964030 5869 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964033 5869 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964036 5869 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964038 5869 flags.go:64] FLAG: --lock-file="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964041 5869 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964044 5869 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964047 5869 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964052 5869 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964055 5869 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964058 5869 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964061 5869 flags.go:64] FLAG: --logging-format="text" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964064 5869 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964067 5869 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964070 5869 flags.go:64] FLAG: --manifest-url="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964072 5869 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964077 5869 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:37:55.971881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964080 5869 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964084 5869 flags.go:64] FLAG: --max-pods="110" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964087 5869 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964107 5869 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964111 5869 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964114 5869 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964117 5869 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964119 5869 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964122 5869 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964129 5869 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964132 5869 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964136 5869 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964139 5869 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964141 5869 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964148 5869 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964152 5869 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964154 5869 flags.go:64] FLAG: --port="10250" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964157 5869 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964160 5869 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f0aacc8f40e0bf79" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964164 5869 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964167 5869 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964170 5869 flags.go:64] FLAG: --register-node="true" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964172 5869 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964176 5869 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964179 5869 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964182 5869 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:37:55.972511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964185 5869 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964189 5869 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964192 5869 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964195 5869 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964198 5869 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964200 5869 flags.go:64] FLAG: --runonce="false" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964204 5869 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964206 5869 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964209 5869 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964212 5869 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964215 5869 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964219 5869 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964222 5869 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964225 5869 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964227 5869 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964231 5869 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964233 5869 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964236 5869 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964239 5869 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964243 5869 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964247 5869 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964252 5869 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964255 5869 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964257 5869 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964261 5869 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:37:55.973170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964264 5869 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964267 5869 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964269 5869 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964272 5869 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964275 5869 flags.go:64] FLAG: --v="2" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964279 5869 flags.go:64] FLAG: --version="false" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964283 5869 flags.go:64] FLAG: --vmodule="" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964286 5869 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964290 5869 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964419 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964423 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964426 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964428 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964431 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964434 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964436 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964439 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964442 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964444 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964447 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964449 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.973987 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964452 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964454 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964457 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964459 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964462 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964466 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964469 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964471 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964474 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964476 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964479 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964481 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964484 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964486 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964489 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964491 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964494 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964496 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964498 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964501 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.974641 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964504 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964506 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964509 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964512 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964514 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964517 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964519 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964522 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964524 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964527 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964529 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964532 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964535 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964537 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964541 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964544 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964547 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964552 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964554 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.975452 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964557 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964559 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964562 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964564 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964567 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964569 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964572 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964574 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964577 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964579 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964582 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964584 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964587 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964589 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964592 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964595 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964597 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964600 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964602 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964605 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.975947 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964607 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964610 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964612 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964615 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964617 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964620 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964622 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964624 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964627 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964630 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964633 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964636 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964638 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964641 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964643 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964646 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964649 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964651 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964654 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964656 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.976449 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964667 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964672 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964675 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964677 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964682 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964685 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964687 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964690 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964693 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964696 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964698 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964701 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964703 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964706 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964708 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.977028 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.964734 5869 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964766 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964770 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964773 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964775 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964778 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964780 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964784 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964787 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964790 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964793 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964795 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964798 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964801 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964803 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964806 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964808 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964811 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964814 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964816 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.977532 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964819 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964821 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964825 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964827 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964830 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964833 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964836 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964838 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964841 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964843 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964846 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964848 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964851 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964853 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964856 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964858 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964861 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964863 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964865 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964869 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.978154 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964872 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964874 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964877 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964879 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964882 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964884 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964887 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964890 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964892 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964895 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964898 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964900 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964903 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964905 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964911 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964915 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964918 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964921 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964923 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.978637 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964927 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964929 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964932 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964934 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964937 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964939 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964942 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964944 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964946 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964949 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964952 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964954 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964958 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964961 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964963 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964966 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964968 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964971 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964973 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964976 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.979108 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964978 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964981 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964983 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964985 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964988 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964990 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964993 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.964997 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965000 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965002 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965005 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965008 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965011 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965013 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965016 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965018 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965021 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965023 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965026 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965028 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.979609 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965031 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965035 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965039 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965042 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965046 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965049 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965051 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.965054 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.968625 5869 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.968639 5869 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.968665 5869 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.968671 5869 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968726 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968732 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968735 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968738 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968741 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968744 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.980203 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968747 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968750 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968753 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968756 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968758 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968761 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968764 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968767 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968770 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968772 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968775 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968778 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968781 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968785 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968790 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968793 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968796 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968800 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968803 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.980720 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968805 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968809 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968812 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968815 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968818 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968820 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968823 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968826 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968828 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968831 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968833 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968836 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968838 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968841 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968843 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968846 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968848 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968851 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968853 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968856 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.981434 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968859 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968862 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968865 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968867 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968871 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968873 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968876 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968878 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968881 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968883 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968886 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968889 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968891 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968893 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968896 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968899 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968903 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968906 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968909 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968911 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.982169 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968914 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968916 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968919 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968921 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968924 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968927 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968929 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968931 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968934 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968937 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968939 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968942 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968944 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968947 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968949 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968952 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968955 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968958 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968961 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.982903 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968963 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968966 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968968 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968971 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968973 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968976 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968978 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968980 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968983 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968985 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968988 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968990 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968993 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968995 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.968997 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969000 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969003 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969005 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969008 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969010 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.983388 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969012 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969015 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.969041 5869 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969188 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969193 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969196 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969199 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969202 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969205 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969207 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969212 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969216 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969219 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969222 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969224 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.983860 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969227 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969229 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969232 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969234 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969237 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969240 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969242 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969245 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969247 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969249 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969252 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969254 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969257 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969259 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969262 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969264 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969267 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969269 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969272 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969274 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.984279 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969277 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969279 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969282 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969284 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969286 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969289 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969292 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969295 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969298 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969300 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969303 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969305 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969308 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969310 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969313 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969315 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969318 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969320 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969324 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.984763 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969327 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969331 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969333 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969336 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969339 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969342 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969344 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969347 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969349 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969352 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969354 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969356 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969359 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969361 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969365 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969367 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969369 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969372 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969374 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.985230 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969380 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969382 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969385 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969387 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969390 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969392 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969395 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969397 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969399 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969402 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969404 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969407 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969409 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969412 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969414 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969417 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969419 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969421 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969424 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.985685 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969426 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969429 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969431 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969434 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969436 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969439 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969441 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969444 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969446 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969449 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969451 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969454 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969456 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969460 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969463 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969466 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.986199 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969469 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.969491 5869 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969524 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969527 5869 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969530 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969533 5869 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969536 5869 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969539 5869 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969541 5869 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969544 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969547 5869 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969551 5869 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969554 5869 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969556 5869 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969559 5869 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969561 5869 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:55.986586 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969564 5869 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969566 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969569 5869 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969573 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969576 5869 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969578 5869 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969581 5869 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969583 5869 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969586 5869 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969588 5869 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969591 5869 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969593 5869 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969596 5869 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969600 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969602 5869 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969605 5869 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969608 5869 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969611 5869 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969614 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969616 5869 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:55.987004 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969619 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969621 5869 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969624 5869 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969626 5869 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969628 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969631 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969634 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969636 5869 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969639 5869 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969641 5869 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969644 5869 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969647 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969650 5869 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969654 5869 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969656 5869 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969660 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969663 5869 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969666 5869 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969670 5869 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:55.987534 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969673 5869 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969676 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969679 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969682 5869 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969684 5869 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969687 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969690 5869 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969693 5869 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969696 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969699 5869 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969701 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969704 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969706 5869 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969709 5869 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969711 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969714 5869 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969716 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969719 5869 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969721 5869 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:55.988054 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969724 5869 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969726 5869 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969729 5869 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969731 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969734 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969736 5869 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969739 5869 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969741 5869 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969744 5869 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969748 5869 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969750 5869 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969753 5869 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969755 5869 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969758 5869 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969761 5869 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969763 5869 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969766 5869 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969768 5869 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969771 5869 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:55.988778 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969774 5869 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969777 5869 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969780 5869 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969782 5869 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969785 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969789 5869 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969791 5869 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969794 5869 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969796 5869 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969799 5869 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969801 5869 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969803 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969806 5869 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969808 5869 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:37:55.969811 5869 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.969857 5869 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.971830 5869 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.971891 5869 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.972122 5869 server.go:1014] "Starting client certificate rotation" Oct 30 18:37:55.989570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.972265 5869 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.972316 5869 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:43 +0000 UTC" deadline="2028-08-03 17:29:27.673816202 +0000 UTC" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.972350 5869 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15430h51m31.701468013s" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.972602 5869 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.973702 5869 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.974935 5869 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.976849 5869 log.go:25] "Validated CRI v1 image API" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.977597 5869 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:37:55.990246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.979747 5869 fs.go:135] Filesystem UUIDs: map[6cff51bd-09d2-4299-9ad1-9febfd51eb5b:/dev/nvme0n1p4 70237958-2b29-40d7-9b22-f2916fd4c3e8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Oct 30 18:37:55.990527 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.979770 5869 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a/userdata/shm major:0 minor:476 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05/userdata/shm major:0 minor:306 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a/userdata/shm major:0 minor:261 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb/userdata/shm major:0 minor:477 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725/userdata/shm major:0 minor:97 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2/userdata/shm major:0 minor:347 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230/userdata/shm major:0 minor:480 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c/userdata/shm major:0 minor:263 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb/userdata/shm major:0 minor:438 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe/userdata/shm major:0 minor:255 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05a02e6e-236c-4782-95af-7e99fa284781/volumes/kubernetes.io~projected/kube-api-access-fjpjb:{mountpoint:/var/lib/kubelet/pods/05a02e6e-236c-4782-95af-7e99fa284781/volumes/kubernetes.io~projected/kube-api-access-fjpjb major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~projected/kube-api-access-48fqm:{mountpoint:/var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~projected/kube-api-access-48fqm major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~secret/metrics-certs major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/182b13f1-6909-4251-8455-eed4d52ae204/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/182b13f1-6909-4251-8455-eed4d52ae204/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:474 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~projected/kube-api-access-jfk42:{mountpoint:/var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~projected/kube-api-access-jfk42 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/alertmanager/9 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/config-reloader/6 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~empty-dir/config-out major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/kube-api-access-x8flf:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/kube-api-access-x8flf major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/tls-assets major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/config-volume major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/web-config major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~projected/kube-api-access-kvck9:{mountpoint:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~projected/kube-api-access-kvck9 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7190032a-e51d-4bd0-95e5-30f6f95e59a4/volumes/kubernetes.io~projected/kube-api-access-9h9mm:{mountpoint:/var/lib/kubelet/pods/7190032a-e51d-4bd0-95e5-30f6f95e59a4/volumes/kubernetes.io~projected/kube-api-access-9h9mm major:0 minor:475 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7268393d-9be4-4e1c-87ff-c9ac7cfd7072/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7268393d-9be4-4e1c-87ff-c9ac7cfd7072/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~projected/kube-api-access-kd2dc:{mountpoint:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~projected/kube-api-access-kd2dc major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~projected/kube-api-access-7sg2c:{mountpoint:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~projected/kube-api-access-7sg2c major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:254 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~projected/kube-api-access-5qpzt:{mountpoint:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~projected/kube-api-access-5qpzt major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~projected/kube-api-access-98jlw:{mountpoint:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~projected/kube-api-access-98jlw major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bf1866b-1e63-4886-830d-c8645a8e5ff5/volumes/kubernetes.io~projected/kube-api-access-n6rd6:{mountpoint:/var/lib/kubelet/pods/8bf1866b-1e63-4886-830d-c8645a8e5ff5/volumes/kubernetes.io~projected/kube-api-access-n6rd6 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~projected/kube-api-access-v562v:{mountpoint:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~projected/kube-api-access-v562v major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~projected/kube-api-access-qh6sh:{mountpoint:/var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~projected/kube-api-access-qh6sh major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~projected/kube-api-access-lgjk5:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~projected/kube-api-access-lgjk5 major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/config-reloader/2 major:0 minor:432 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:432 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/prometheus/15 major:0 minor:432 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~empty-dir/config-out major:0 minor:422 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/kube-api-access-6xjjz:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/kube-api-access-6xjjz major:0 minor:437 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/tls-assets major:0 minor:433 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/config major:0 minor:430 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:428 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:435 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:429 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:427 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:431 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/web-config major:0 minor:432 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4e82816-cfc3-413b-a254-47dd3a2535ea/volumes/kubernetes.io~projected/kube-api-access-tv87d:{mountpoint:/var/lib/kubelet/pods/e4e82816-cfc3-413b-a254-47dd3a2535ea/volumes/kubernetes.io~projected/kube-api-access-tv87d major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~projected/kube-api-access-w6cc6:{mountpoint:/var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~projected/kube-api-access-w6cc6 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7/volumes/kubernetes.io~projected/kube-api-access-2cll7:{mountpoint:/var/lib/kubelet/pods/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7/volumes/kubernetes.io~projected/kube-api-access-2cll7 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~projected/kube-api-access-4g84q:{mountpoint:/var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~projected/kube-api-access-4g84q major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0e02e4a-2151-4339-93de-09057afd8f9d/volumes/kubernetes.io~projected/kube-api-access-lg47z:{mountpoint:/var/lib/kubelet/pods/f0e02e4a-2151-4339-93de-09057afd8f9d/volumes/kubernetes.io~projected/kube-api-access-lg47z major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4cc7620-49dc-4d5a-8343-411e0d17309d/volumes/kubernetes.io~projected/kube-api-access-sgwsp:{mountpoint:/var/lib/kubelet/pods/f4cc7620-49dc-4d5a-8343-411e0d17309d/volumes/kubernetes.io~projected/kube-api-access-sgwsp major:0 minor:65 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/48a33a64624cbac20b022a9c5a49a2759edcbbfb41255e8d987f5725e509fe4f/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/d510a510d958dd666d716ff7af3c57fc1d831e5ad8e59ac13945565182b4ac6f/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/61af3524a5ea9c88cda38cfc34b9004a8ccfa0964d412f933559efefced51049/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/81fdd31f26849e800ec26fb77530e3857ac1c65ed6aa907efcaa5e17afcf8a1b/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/8dd5a7b63586fb09ad0a3c2d77496528b2000cad0687727708cb6b28b89c8720/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/debc2c46cd0179e484465302fead791f880ff90ef828e80a820919aa4981c51d/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/9cdaa36db6e0ab706072943668b49067b9e2ff19b7fd83cec1c3ebb01cebedcb/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/13ed093d57b938caaffbb6c4faa8324809cc88f88b4a136f83cf38ab40ae387e/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/02d51e3f37ca2141911c06d6f736093f760b66fc8afcc7b35b9e243f6fbff065/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/af01754ca5e25085da2de91ab4936b2b32e1e51ba9d49c73644200d9e7f3e12f/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/ed3e9ade293850843f4ac780f2d11584b18228077810e1bcb93aa112466d892a/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/d883068172d49f3d7864d3aa67454b561bb78da8df40750b30e3be74bf3ca8e9/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/3c13cfc6dff4a9bebfe715faf4240dca6d34bc2f7e281fa46e962fc014e71594/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/4028519c121b195da557f88d712983328d57dd2d82578684cab2606450e4f483/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/31faa8f323f438d89120b526be4b341a9d60de661845c1717116e160507714b5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/f175c09ebd51f780e3f4ddc0396594f6ec3c0587ae3bf758f6716daccd680c69/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/e29938bb499f0d4aed6183d51e64575aead129bae37547ece04ef9834abd4f1c/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/893a1031eb4ab304dfb4710fed53b80f126dde3db69ec1e8399c60d158cd707f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/009b96ffd842ee444d9c17a119d9c4fad1070874f8c4c054ad582b01b60a8e88/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c370ed850c7145c3b42fa7c95f19e1138778fa4a51ff1e10fabd2447c401f71c/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b5fb60f82850fd646d014a2d0ad068d387835f478e4494c6fc3419d0e4656ac2/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/99b0f4aa9721286681c6e1311e39d5e462b4e6d81a4b754f78d62dbb5253594b/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/8154d629d9d1c209c1ea6d76a4e9f264b309165add6ccebf2856b3fada772bee/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/8841d7210cadff6bfd09e8f144f58aba8df8bd20a6a19cf3a3a202ada89a8a8d/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/6696e0912392384d1bfd744c987aba5ee075e28bb24573e21b668f99b1b95a81/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/2dd957efe71ef9ef8d8ca0f5ebcde13ac0c7397478882ee92925b6f7735e46ad/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/8d175387f75294da90ba1844975924d1da240268ea479e51ba5b539b8ce48ff8/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/bd6471fc5abf870640c195eaa2b6ae5f41eb28aa94363fb7108152921dc206fe/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/36256f74760cd1f9e50337774cd3ac25541cc23abb29f0757c51978bb8dd6936/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/afdf9cc70a302590415bcff7b340d6a2abedb0df0b2e5f44fe84ba553e15aea4/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/377d486674f8f05c5be1ae8539b28bbbb3b69db8061cdb605d3d116c03c6ae3d/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/e08fe99fdfd94744e2ea0f1c7dcb16818607ea14ba8f1a1758a11f7091fe2429/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/a63703ebc88766d5e0470762100f8952a15499056a51cbf5c5f3d287309c5be3/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/8fc140e01b19634d2f48a56508fd6d5ee021b871b2851bc307e74402a5e7044f/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/090d62f7383efe086acb42bbaf0905f5b1cac20e41c9723fef8ab9f80e0cf853/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/91d6bb6cdd6eff600a3d407da7280a83377e180286e5b04320cb117ca8717536/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/73742bdbe1b10718abf6a5cb4197426f5dc3d86eb0684d3a57d7fa958336fb87/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/30571cfb2d347934841d108399a3cf885b356eb137824ad2c01a7fcb08e9b704/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/4e4f8cdabfc407ab74629a2bc4609d42be5d681fa0b084f37c9e1242cbb49c4e/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/b42aed38a54e4bb8f037b96fd0349595ce08b73802bfda1c96e83685584f1383/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/aa20b263450308853607e7547cba2d45cdd3c3c3906f1eee668b953d4a6d4c34/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/be1f951085f597e65fa4b08ff02827ebcd3a6c1cda91c4907697c406a9e0db9b/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/7ca8fc432c811b6478ecab8649ea778f445c4ed8d1d4da909563ac6995a74135/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/88c73ba7196a9ac7012f07c78b14cd74c02ce4d5c884de732b18c93f76d3fce6/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/e48bbd4528a2b0b1581106ecd8eb232a333667726a33dfff635d652867f74a50/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/4f2fcadf91266fb4424d6f3a3e48b11704dce44cdd62a12c2e914d7de979b385/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/7a7e9a9a7e1dcad46cfdb8ec4bed7ae11c651608ad8648a1691a78ea9ed1f09f/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/dc706384c9c737c90bb2b6caf39c19cf91a7e7fe717bd946e28897d8790e00b5/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/510f6ccdb4f5ef4cb27b365d8abdc6249ef780862807da30c154a7e4d6806c63/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/5a0595220d3befc0b19fb546f8fb11306d35f17b7d15c22fa793df5c49a44421/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/1a4a5639da7edb41600783417ee3570d37d790129ce0ab5b8975952033b36747/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-382:{mountpoint:/var/lib/containers/storage/overlay/90531437779c1f2f94372491d955098f94522a27a3ec4cc1242d76a93e693383/merged major:0 minor:382 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/6b72454ae206277121aaec246a400c620d89ed9065cda42d7e4d001d3ed2ac0e/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/360d1c364800d00d270315483e545a7ff4a0ba6f74285c801f8d5af094745b21/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/8f864fd71596a24392d960701d73031bfdfa77df212d7ee8e074e2656dc3b895/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/db1e4c340b771b4572e1b8289c782b912ebfb24bfbe1695d733fad21f133aaa5/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/877bb0bde741d888c095cb7673df906016c6e6bcb73081e01eb308e7c8385103/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-412:{mountpoint:/var/lib/containers/storage/overlay/7e1fa94bb5b58757f94d15fc030a146d5acdf23ef0a383aa12ea267f1e2d9957/merged major:0 minor:412 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/27deced38c64c139d292a86cdef19a8dc25ccfbe74a91eacef766b81cf92a81b/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/5f9c1876a5b8748ab2fda1a4da1cce1678a637e833697566a3eae288ea5507a6/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/7bf27a57440bc82a76848289b4e4db910057c1dc3c5717e7ebc701dfd65b2f5c/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-444:{mountpoint:/var/lib/containers/storage/overlay/9e3e08081a86661436a5880e3b669764939fa3ff901bfb0ea9b4055170979681/merged major:0 minor:444 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/17ca9ae6c4192aad7524b128584546e213fe428ede1dba3d0e5bd1d6eed0c7cb/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-455:{mountpoint:/var/lib/containers/storage/overlay/2da5eb9a02dca57162f48f6541c9fe98107cf687b8cb1e77cdce14f2300fb367/merged major:0 minor:455 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/bc300371e3fc78d1511391b3774bbead5922d19702c9690b35a8699e75849034/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-465:{mountpoint:/var/lib/containers/storage/overlay/b99ee794128cbf03e0282c20f79f0f3c691c9dfd8221fd9425245cce8f2cb766/merged major:0 minor:465 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/596b273463bfce51c1d5364e6d360ef563cd581c4a41eb741df0c4093a12001f/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-484:{mountpoint:/var/lib/containers/storage/overlay/dc4fcc0b4768e61abd2c8446f6a4e03f2f8c8eabdd0505c14927749464b26f1c/merged major:0 minor:484 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/bd2cc15a8d0a4bc6d0077888b1535e02a36f51a5b800eba9c145f6df6f1d91be/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-488:{mountpoint:/var/lib/containers/storage/overlay/fbeb8d0761487d2c67f1f0fcf15161a5e86536ee0127604da94707938b3ee0fd/merged major:0 minor:488 fsType:overlay blockSize:0} overlay_0-490:{mountpoint:/var/lib/containers/storage/overlay/14341e7c9679963b336b35bd53bca278795e5bc56b01f9e5842148144574ae48/merged major:0 minor:490 fsType:overlay blockSize:0} overlay_0-496:{mountpoint:/var/lib/containers/storage/overlay/ef71f60b699c5d43c3b4add801af6e6fdcde3239a085eb5d645120f69c5636f4/merged major:0 minor:496 fsType:overlay blockSize:0} overlay_0-501:{mountpoint:/var/lib/containers/storage/overlay/b59a9b3e6f1a6768691954825bb78d3b879963ef6cb1ed03d447a2d92987bac0/merged major:0 minor:501 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/0ce6f3bf44d664a4c5c4eb8f5e5d69497459df63c6f64b19e8ed4cfe8413153c/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/2306f4377823e3cb9559c00caeb46abf6bbb19c88b95814f4ed9671c238da588/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/b013b919dcbc7b964485cab449f16bd3e4a39f073c2f81b723a241caa4021aa9/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/fa2533403c5dafc83489bc9c43ad5e8d28627390a7290f9ed97a4ffbd9e98163/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/65cd87e4e282d547ff2571779f531a37ac9fb142bb18d94c9d345b18c89da3b7/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ec479d7516cb57676b122b8f2e2f419f3deb92c043138501c79e0c03b64d097f/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/9a0dfd306853cac07e995e874fa64242c53e9395344394eb9938331a1749f456/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/329eef7de551191f09b9298feaef4d0614767f3553eb5883d950104e21c82ae1/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/0b075df755b1611e3da4abd742cec36b965436e1d2d5b050933a1dca166d9282/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/5a616eee595d2a465e919ef16f97645f21a68fc930d45f5045e09b4dc90c692a/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.987162 5869 manager.go:217] Machine: {Timestamp:2026-10-30 18:37:55.986700145 +0000 UTC m=+0.058524212 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100110 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec290e98ecedbf2334c34433c794980b SystemUUID:ec290e98-eced-bf23-34c3-4433c794980b BootID:cafc8689-f743-478b-848f-1ed79bb9faac Filesystems:[{Device:/var/lib/kubelet/pods/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7/volumes/kubernetes.io~projected/kube-api-access-2cll7 DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d5dae3dbd0de951b93c022d4e9111cb2147fcbf49889c1784d55c4f5c7d7870/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:253 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9f9242bc1930e16f4038a33419819f8fd4d0c3fc11055663a2cc6ab8e07c9d2c/userdata/shm DeviceMajor:0 DeviceMinor:263 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:289 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:340 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f0e02e4a-2151-4339-93de-09057afd8f9d/volumes/kubernetes.io~projected/kube-api-access-lg47z DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/efcce32a15703b386a06b997cab9f9dc3215bbab2efe9e37a2d603ae08170a39/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-382 DeviceMajor:0 DeviceMinor:382 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89a32d56d8caabcb33e62c2019435cd0d2f65caec5f28e6b8498875f06b14230/userdata/shm DeviceMajor:0 DeviceMinor:480 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-496 DeviceMajor:0 DeviceMinor:496 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2152bfaa74f2dde38c94fa111f625ef146651af36b20826fa4079078b884335c/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:288 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/kube-api-access-x8flf DeviceMajor:0 DeviceMinor:291 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:430 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b925069901ce1e49c53b8919ad247d3b20ff9cc75cff0f99aa980e9497800193/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26dca8ec4f3961a9943496d1fe447f277fdcd2ed298b3095c0ebbf9fb5c451f0/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-501 DeviceMajor:0 DeviceMinor:501 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:251 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~projected/kube-api-access-v562v DeviceMajor:0 DeviceMinor:259 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:432 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:335 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7fb808b0777666749923dbf6b5d3f2926ebcb5c0bf7d42b476a52843234a5186/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:435 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-455 DeviceMajor:0 DeviceMinor:455 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-465 DeviceMajor:0 DeviceMinor:465 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-490 DeviceMajor:0 DeviceMinor:490 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~projected/kube-api-access-5qpzt DeviceMajor:0 DeviceMinor:252 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:286 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7190032a-e51d-4bd0-95e5-30f6f95e59a4/volumes/kubernetes.io~projected/kube-api-access-9h9mm DeviceMajor:0 DeviceMinor:475 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:427 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e70452c4-c454-4edd-857c-3eb8d36acd12/volumes/kubernetes.io~projected/kube-api-access-w6cc6 DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:303 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f90617652e81472a26a2f6e9669e2496254b5f13a4a67fab3fda3bb0a79ebeb/userdata/shm DeviceMajor:0 DeviceMinor:477 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:290 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9211d49d5cbf559270625c548b566387a000e66608a2d3b5b68c75107d4a3766/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:341 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:428 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~projected/kube-api-access-kd2dc DeviceMajor:0 DeviceMinor:343 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/779283349b8d52b4f32a1936d8ccff745aa6b28f8797545052b0f114d7550c2e/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:257 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:301 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:304 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ef44a6cf-b792-4c9c-8503-271d365976f8/volumes/kubernetes.io~projected/kube-api-access-4g84q DeviceMajor:0 DeviceMinor:339 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~projected/kube-api-access-48fqm DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e4e82816-cfc3-413b-a254-47dd3a2535ea/volumes/kubernetes.io~projected/kube-api-access-tv87d DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7a6523be-9435-46c7-b5a2-b84a2bba0d83/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:241 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/772108aff3c750f0d6f95fe1341cda28a5f828022fc5e63f0c6f0008ee4581b2/userdata/shm DeviceMajor:0 DeviceMinor:347 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:434 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:432 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~projected/kube-api-access-98jlw DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:283 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~projected/kube-api-access-lgjk5 DeviceMajor:0 DeviceMinor:296 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/211dfbe3bd0359772659a3537777b3815b7a6cf9995fe9c01264f177293b1d05/userdata/shm DeviceMajor:0 DeviceMinor:306 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb/userdata/shm DeviceMajor:0 DeviceMinor:438 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/11de6f5cffcbcf03b1a02766c59c4783d1764cb7696997efb3bd2d04e75bb28a/userdata/shm DeviceMajor:0 DeviceMinor:476 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4aedc1741a475925cb60b98788c2c17815f071cf27106750c2a08c36a88a0725/userdata/shm DeviceMajor:0 DeviceMinor:97 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-484 DeviceMajor:0 DeviceMinor:484 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52ca3b9e-8619-467a-a705-5bcbabdd7a08/volumes/kubernetes.io~projected/kube-api-access-kvck9 DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa596b8bf0563ca9dcff5a0248a82f903c3d3cdf4b2859bbc7638c91761ebe97/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab46616b-1670-40b4-97be-689b9931022c/volumes/kubernetes.io~projected/kube-api-access-qh6sh DeviceMajor:0 DeviceMinor:208 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:429 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:432 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:422 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/182b13f1-6909-4251-8455-eed4d52ae204/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:474 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2884b7ce3906a26630c912e977c45984cee5cd5f68f99408e037395aaeab6f0a/userdata/shm DeviceMajor:0 DeviceMinor:261 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-488 DeviceMajor:0 DeviceMinor:488 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05a02e6e-236c-4782-95af-7e99fa284781/volumes/kubernetes.io~projected/kube-api-access-fjpjb DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:285 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:285 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:433 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~projected/kube-api-access-6xjjz DeviceMajor:0 DeviceMinor:437 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-444 DeviceMajor:0 DeviceMinor:444 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7f5e5aef-db22-432b-9bfc-64231443a33c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7268393d-9be4-4e1c-87ff-c9ac7cfd7072/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9086307a1a43f2f65a0ba762f4fff045a63bd883a8b7c36388db4ecda25f5bff/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:431 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/166168ae-19c0-4650-9a99-be2f63f2917f/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:470 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f4cc7620-49dc-4d5a-8343-411e0d17309d/volumes/kubernetes.io~projected/kube-api-access-sgwsp DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada/volumes/kubernetes.io~projected/kube-api-access-jfk42 DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a1df3f31-27af-4f32-aa1e-e07cf031f2cf/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:256 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:284 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f917329ba616055a7a3cd41a2a755ed46ca271bce90e57c8e988747c12dd8dbe/userdata/shm DeviceMajor:0 DeviceMinor:255 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:305 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~projected/kube-api-access-7sg2c DeviceMajor:0 DeviceMinor:260 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/79e792e9-28c1-4127-b364-3679fcf27c7c/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:254 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:300 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af2c013dc3e2c14f42d2450fce70e1591c80f6345e8ccc140b32caa2b72fcf69/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdde0beb-f496-4013-aff3-3706e199e7f2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:302 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:436 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8bf1866b-1e63-4886-830d-c8645a8e5ff5/volumes/kubernetes.io~projected/kube-api-access-n6rd6 DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/734c247a-5ffb-4ccc-9720-8ba9d5ac3711/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:342 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-412 DeviceMajor:0 DeviceMinor:412 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:432 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:11de6f5cffcbcf0 MacAddress:5e:23:84:4e:44:4e Speed:10000 Mtu:8901} {Name:211dfbe3bd03597 MacAddress:42:0b:b9:3e:8a:88 Speed:10000 Mtu:8901} {Name:2152bfaa74f2dde MacAddress:42:90:d6:fa:ec:ee Speed:10000 Mtu:8901} {Name:2884b7ce3906a26 MacAddress:86:83:ce:b6:e5:41 Speed:10000 Mtu:8901} {Name:2f90617652e8147 MacAddress:d2:b7:a0:b5:26:35 Speed:10000 Mtu:8901} {Name:6d5dae3dbd0de95 MacAddress:42:cf:20:d9:98:9f Speed:10000 Mtu:8901} {Name:772108aff3c750f MacAddress:a6:36:bd:25:dd:bd Speed:10000 Mtu:8901} {Name:7fb808b07776667 MacAddress:de:28:e3:c8:f4:36 Speed:10000 Mtu:8901} {Name:89a32d56d8caabc MacAddress:72:d6:5e:ef:70:c4 Speed:10000 Mtu:8901} {Name:9086307a1a43f2f MacAddress:b6:0c:74:17:da:4b Speed:10000 Mtu:8901} {Name:a7db8e500812074 MacAddress:36:fb:b2:42:66:da Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:cb:dd:7f:0b Speed:0 Mtu:9001} {Name:br-int MacAddress:42:71:45:03:17:f9 Speed:0 Mtu:8901} {Name:d485655269704b2 MacAddress:3e:8f:0a:60:5b:b7 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:cb:dd:7f:0b Speed:0 Mtu:9001} {Name:f917329ba616055 MacAddress:02:34:6f:ed:d3:63 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:7a:33:aa:e4:a3:6e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:fe:c9:2d:22:35:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.987607 5869 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.987646 5869 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.987875 5869 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.987894 5869 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-38.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988003 5869 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988010 5869 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988024 5869 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988036 5869 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988115 5869 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988262 5869 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988366 5869 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988400 5869 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988413 5869 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988431 5869 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988444 5869 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.988455 5869 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.990622 5869 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:37:55.990833 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.990633 5869 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:55.991641 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.991074 5869 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:37:55.993626 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.993611 5869 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:37:55.993669 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.993626 5869 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:55.993739 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.993729 5869 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:37:55.993916 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.993904 5869 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:37:55.993944 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.993935 5869 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:37:55.994031 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994022 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:37:55.994060 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994042 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:37:55.994060 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994052 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994062 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994071 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994080 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994085 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994110 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994117 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:37:55.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994124 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:37:55.994319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994137 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:37:55.994319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994148 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:37:55.994319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994188 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:37:55.994319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.994195 5869 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:37:55.997536 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.997523 5869 server.go:1263] "Started kubelet" Oct 30 18:37:55.997664 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.997627 5869 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:37:55.997722 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.997655 5869 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:37:55.997780 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.997737 5869 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:37:55.998010 ip-10-0-128-38 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:37:55.998182 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.998047 5869 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:37:55.999055 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:55.999036 5869 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:37:56.001021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.000994 5869 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:37:56.001473 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001066 5869 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:37:56.001556 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001479 5869 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:37:56.001556 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.001146 5869 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:37:56.001556 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001127 5869 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:37:56.001556 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001493 5869 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:37:56.001773 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001540 5869 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:45 +0000 UTC" deadline="2028-07-14 20:15:36.526881428 +0000 UTC" Oct 30 18:37:56.001773 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.001571 5869 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14953h37m40.525313936s" Oct 30 18:37:56.004854 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.004830 5869 factory.go:55] Registering systemd factory Oct 30 18:37:56.004948 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.004855 5869 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:37:56.005229 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005209 5869 factory.go:153] Registering CRI-O factory Oct 30 18:37:56.005229 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005227 5869 factory.go:223] Registration of the crio container factory successfully Oct 30 18:37:56.005361 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005292 5869 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:37:56.005361 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005311 5869 factory.go:103] Registering Raw factory Oct 30 18:37:56.005361 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005324 5869 manager.go:1199] Started watching for new ooms in manager Oct 30 18:37:56.005746 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.005730 5869 manager.go:319] Starting recovery of all containers Oct 30 18:37:56.007745 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007694 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:37:56.007849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007753 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab46616b-1670-40b4-97be-689b9931022c" volumeName="kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh" seLinuxMountContext="" Oct 30 18:37:56.007849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007779 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:37:56.007849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007795 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="166168ae-19c0-4650-9a99-be2f63f2917f" volumeName="kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs" seLinuxMountContext="" Oct 30 18:37:56.007849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007810 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out" seLinuxMountContext="" Oct 30 18:37:56.007849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007836 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007853 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007876 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bf1866b-1e63-4886-830d-c8645a8e5ff5" volumeName="kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007899 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4cc7620-49dc-4d5a-8343-411e0d17309d" volumeName="kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007923 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007938 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7190032a-e51d-4bd0-95e5-30f6f95e59a4" volumeName="kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007974 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.007996 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008012 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008039 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008053 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4cc7620-49dc-4d5a-8343-411e0d17309d" volumeName="kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config" seLinuxMountContext="" Oct 30 18:37:56.008176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008074 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4cc7620-49dc-4d5a-8343-411e0d17309d" volumeName="kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008334 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="166168ae-19c0-4650-9a99-be2f63f2917f" volumeName="kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008383 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca3b9e-8619-467a-a705-5bcbabdd7a08" volumeName="kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008399 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008419 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008438 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008457 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008469 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008480 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008548 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008564 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008582 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs" seLinuxMountContext="" Oct 30 18:37:56.008648 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008624 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008642 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bf1866b-1e63-4886-830d-c8645a8e5ff5" volumeName="kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008692 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008725 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008737 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008751 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3740ec35-b3ca-4b78-a3f2-610eeb7b3ada" volumeName="kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008830 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008848 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008953 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.008992 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009073 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bf1866b-1e63-4886-830d-c8645a8e5ff5" volumeName="kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:37:56.009203 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009146 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab46616b-1670-40b4-97be-689b9931022c" volumeName="kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009224 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009253 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7268393d-9be4-4e1c-87ff-c9ac7cfd7072" volumeName="kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009340 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009395 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a6523be-9435-46c7-b5a2-b84a2bba0d83" volumeName="kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009471 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab46616b-1670-40b4-97be-689b9931022c" volumeName="kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009507 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009589 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out" seLinuxMountContext="" Oct 30 18:37:56.009644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009619 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7" volumeName="kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009655 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7" volumeName="kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009673 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3740ec35-b3ca-4b78-a3f2-610eeb7b3ada" volumeName="kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009752 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009791 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009841 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009903 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a6523be-9435-46c7-b5a2-b84a2bba0d83" volumeName="kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls" seLinuxMountContext="" Oct 30 18:37:56.009945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009937 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Oct 30 18:37:56.010180 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.009952 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db" seLinuxMountContext="" Oct 30 18:37:56.010180 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010036 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config" seLinuxMountContext="" Oct 30 18:37:56.010180 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010078 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734c247a-5ffb-4ccc-9720-8ba9d5ac3711" volumeName="kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.010180 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010156 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f5e5aef-db22-432b-9bfc-64231443a33c" volumeName="kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp" seLinuxMountContext="" Oct 30 18:37:56.010306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010179 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file" seLinuxMountContext="" Oct 30 18:37:56.010306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010209 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4e82816-cfc3-413b-a254-47dd3a2535ea" volumeName="kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d" seLinuxMountContext="" Oct 30 18:37:56.010306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010229 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0e02e4a-2151-4339-93de-09057afd8f9d" volumeName="kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:37:56.010306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010266 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05a02e6e-236c-4782-95af-7e99fa284781" volumeName="kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb" seLinuxMountContext="" Oct 30 18:37:56.010425 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010295 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls" seLinuxMountContext="" Oct 30 18:37:56.010425 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010317 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca3b9e-8619-467a-a705-5bcbabdd7a08" volumeName="kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:37:56.010425 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010334 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Oct 30 18:37:56.010425 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010390 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets" seLinuxMountContext="" Oct 30 18:37:56.010425 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010408 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz" seLinuxMountContext="" Oct 30 18:37:56.010559 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010436 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef44a6cf-b792-4c9c-8503-271d365976f8" volumeName="kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert" seLinuxMountContext="" Oct 30 18:37:56.010559 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010470 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3740ec35-b3ca-4b78-a3f2-610eeb7b3ada" volumeName="kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume" seLinuxMountContext="" Oct 30 18:37:56.010559 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010485 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7268393d-9be4-4e1c-87ff-c9ac7cfd7072" volumeName="kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs" seLinuxMountContext="" Oct 30 18:37:56.010559 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010507 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a6523be-9435-46c7-b5a2-b84a2bba0d83" volumeName="kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.010559 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010543 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bf1866b-1e63-4886-830d-c8645a8e5ff5" volumeName="kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy" seLinuxMountContext="" Oct 30 18:37:56.010717 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010565 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 18:37:56.010717 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010602 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls" seLinuxMountContext="" Oct 30 18:37:56.010717 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010635 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls" seLinuxMountContext="" Oct 30 18:37:56.010717 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010658 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e70452c4-c454-4edd-857c-3eb8d36acd12" volumeName="kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert" seLinuxMountContext="" Oct 30 18:37:56.010717 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010675 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="182b13f1-6909-4251-8455-eed4d52ae204" volumeName="kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret" seLinuxMountContext="" Oct 30 18:37:56.010864 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010732 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3740ec35-b3ca-4b78-a3f2-610eeb7b3ada" volumeName="kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42" seLinuxMountContext="" Oct 30 18:37:56.010864 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010765 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdde0beb-f496-4013-aff3-3706e199e7f2" volumeName="kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls" seLinuxMountContext="" Oct 30 18:37:56.010864 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010815 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls" seLinuxMountContext="" Oct 30 18:37:56.010991 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010866 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef44a6cf-b792-4c9c-8503-271d365976f8" volumeName="kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q" seLinuxMountContext="" Oct 30 18:37:56.010991 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010902 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca3b9e-8619-467a-a705-5bcbabdd7a08" volumeName="kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides" seLinuxMountContext="" Oct 30 18:37:56.011058 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.010988 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca3b9e-8619-467a-a705-5bcbabdd7a08" volumeName="kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config" seLinuxMountContext="" Oct 30 18:37:56.011058 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011027 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c" seLinuxMountContext="" Oct 30 18:37:56.011163 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011072 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a6523be-9435-46c7-b5a2-b84a2bba0d83" volumeName="kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt" seLinuxMountContext="" Oct 30 18:37:56.011163 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011085 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Oct 30 18:37:56.011163 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011134 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config" seLinuxMountContext="" Oct 30 18:37:56.011163 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011148 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e70452c4-c454-4edd-857c-3eb8d36acd12" volumeName="kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6" seLinuxMountContext="" Oct 30 18:37:56.011340 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011159 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05a02e6e-236c-4782-95af-7e99fa284781" volumeName="kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca" seLinuxMountContext="" Oct 30 18:37:56.011340 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011206 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf" seLinuxMountContext="" Oct 30 18:37:56.011804 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011264 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011813 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79e792e9-28c1-4127-b364-3679fcf27c7c" volumeName="kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011827 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011835 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab46616b-1670-40b4-97be-689b9931022c" volumeName="kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011843 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011856 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011868 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49cb5f50-428f-433b-b372-5b6e19584741" volumeName="kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.011888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011881 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca3b9e-8619-467a-a705-5bcbabdd7a08" volumeName="kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011893 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011905 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d193331c-eb89-4849-b124-76b1bf0d0263" volumeName="kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011936 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0e02e4a-2151-4339-93de-09057afd8f9d" volumeName="kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011949 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f5e5aef-db22-432b-9bfc-64231443a33c" volumeName="kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011961 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f5e5aef-db22-432b-9bfc-64231443a33c" volumeName="kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.011996 5869 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" volumeName="kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v" seLinuxMountContext="" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.012008 5869 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:37:56.012220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.012016 5869 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:37:56.032927 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.032901 5869 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:37:56.034632 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.034609 5869 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:37:56.034632 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.034630 5869 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:37:56.034770 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.034651 5869 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:37:56.034770 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.034696 5869 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:37:56.048759 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048726 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="726f86ac5694fae4f3ded120c54093f54ea40e6864ed185661d99e51063717cc" exitCode=0 Oct 30 18:37:56.048840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048763 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="bb8f9c196f093b3cae73499c1b3c439c3ceba373381ee113ccb450a62c65d05b" exitCode=0 Oct 30 18:37:56.048840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048777 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="d70b742fa196217c551daf91a29659b5b976a683db309efabde5b9d5d48c8c4a" exitCode=0 Oct 30 18:37:56.048840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048791 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="761c5b78d193b1e4187f24081e487d147845f06ee75cfd06c203fa0e6d4afc39" exitCode=0 Oct 30 18:37:56.048840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048803 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="ae5c26a8f8bbcd155a185728eb1097b12ac51c86445fd75862c1c18e274d3db5" exitCode=0 Oct 30 18:37:56.048840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.048815 5869 generic.go:356] "Generic (PLEG): container finished" podID="8bf1866b-1e63-4886-830d-c8645a8e5ff5" containerID="1883c9dafa809ff182d8af27e4a19c1c1e6aed06b8a5d8010d93097378db3513" exitCode=0 Oct 30 18:37:56.055584 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.055565 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" exitCode=0 Oct 30 18:37:56.058017 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.057997 5869 generic.go:356] "Generic (PLEG): container finished" podID="a1df3f31-27af-4f32-aa1e-e07cf031f2cf" containerID="4016e43963013a87dd0665678fa622eb9b8ebd0aa14485c0a277d3b987555170" exitCode=0 Oct 30 18:37:56.076451 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.076428 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" exitCode=0 Oct 30 18:37:56.097348 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.097326 5869 generic.go:356] "Generic (PLEG): container finished" podID="3bec2dc02e02013dddc74f8c61aeafbc" containerID="84bc559fa0bc591105cd035863c805aa961c8b5332aaae183119cadffff4203e" exitCode=0 Oct 30 18:37:56.101563 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.101548 5869 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:37:56.123160 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.123145 5869 manager.go:324] Recovery completed Oct 30 18:37:56.135718 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.135665 5869 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:37:56.143036 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.143021 5869 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:56.143726 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.143713 5869 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:56.146630 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.146616 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:56.146698 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.146642 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:56.146698 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.146652 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:56.149242 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149226 5869 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:56.149297 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149232 5869 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:37:56.149297 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149293 5869 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:37:56.149363 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149313 5869 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:37:56.149464 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149450 5869 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:37:56.149493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149465 5869 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:37:56.149493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149482 5869 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:37:56.149493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149487 5869 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:37:56.149493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149492 5869 policy_none.go:50] "Start" Oct 30 18:37:56.149595 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149502 5869 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:37:56.149595 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149511 5869 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:37:56.149652 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149633 5869 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:37:56.149652 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149642 5869 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:37:56.149652 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.149650 5869 policy_none.go:44] "Start" Oct 30 18:37:56.150173 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.150154 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:56.150261 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.150187 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:56.150261 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.150201 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:56.155723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.155697 5869 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:37:56.158435 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.158421 5869 manager.go:341] "Starting Device Plugin manager" Oct 30 18:37:56.158516 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.158443 5869 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:37:56.158516 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.158451 5869 server.go:87] "Starting device plugin registration server" Oct 30 18:37:56.158703 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.158691 5869 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:37:56.159297 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.158702 5869 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:37:56.159397 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.159323 5869 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:37:56.159455 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.159416 5869 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:37:56.159455 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.159452 5869 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:37:56.159531 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.159458 5869 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:37:56.159586 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.159507 5869 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:37:56.159436796Z","UUID":"cd728ff8-d149-4486-af80-5c7a80ceeff1","Handler":null,"Name":"","Endpoint":""} Oct 30 18:37:56.160646 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.160622 5869 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:37:56.160757 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.160652 5869 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:37:56.160861 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.160839 5869 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:37:56.160964 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.160879 5869 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-38.us-west-2.compute.internal\" not found" Oct 30 18:37:56.238862 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.238840 5869 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:56.260022 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.260005 5869 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:56.261044 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.261028 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:56.261140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.261054 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:56.261140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.261066 5869 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-38.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:56.261140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.261108 5869 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.281202 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.281188 5869 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:56.281324 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.281311 5869 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:37:56.282261 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.282236 5869 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:56.336645 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.336609 5869 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal"] Oct 30 18:37:56.337218 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337202 5869 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.337295 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337260 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"1c6479c60b669fe79980e9a61328bda9","Type":"ContainerStarted","Data":"fb5eb5933f9b98d5bcd27028011c0dc18a7c518c6d9331fe1fa28b23ea07e73f"} Oct 30 18:37:56.337295 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337293 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"1c6479c60b669fe79980e9a61328bda9","Type":"ContainerStarted","Data":"a4749733f743b55e17561e9a30af2c9a77c601870f946eb32c4db2e5a5b533b2"} Oct 30 18:37:56.337395 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337306 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerStarted","Data":"4a548068e9b53ca48ed91bcd7b1154b61d2ea966fd16aa9ca96a6f36434605d3"} Oct 30 18:37:56.337395 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337317 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerDied","Data":"84bc559fa0bc591105cd035863c805aa961c8b5332aaae183119cadffff4203e"} Oct 30 18:37:56.337395 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337330 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" event={"ID":"3bec2dc02e02013dddc74f8c61aeafbc","Type":"ContainerStarted","Data":"30fccac4b7030cbd7a5247d2104747082b3572e9fcd19d312ba2ffbc2d6228c0"} Oct 30 18:37:56.337515 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.337412 5869 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.398022 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.397965 5869 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:56.398022 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.398013 5869 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.399673 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.399660 5869 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:56.399715 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:56.399690 5869 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.405008 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.404992 5869 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.405949 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.405038 5869 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.482846 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.482819 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.482846 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.482845 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.482971 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.482863 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583298 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583276 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583311 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583335 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583383 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583530 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583383 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c6479c60b669fe79980e9a61328bda9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"1c6479c60b669fe79980e9a61328bda9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.583530 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.583384 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bec2dc02e02013dddc74f8c61aeafbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" (UID: \"3bec2dc02e02013dddc74f8c61aeafbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:56.989948 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:56.989918 5869 apiserver.go:52] "Watching apiserver" Oct 30 18:37:57.106867 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.106838 5869 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:57.107030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.107010 5869 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:57.144968 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.144944 5869 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:37:57.145673 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.145646 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-wnj4b","openshift-insights/insights-runtime-extractor-rd622","openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm","openshift-multus/multus-25pf7","openshift-multus/multus-additional-cni-plugins-jnv56","openshift-ovn-kubernetes/ovnkube-node-dbz7h","openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2","openshift-network-operator/iptables-alerter-rpdb6","openshift-cluster-node-tuning-operator/tuned-4xjt2","openshift-monitoring/metrics-server-56686866c6-xhcgv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal","openshift-monitoring/alertmanager-main-0","openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv","openshift-multus/network-metrics-daemon-sndlp","openshift-dns/dns-default-7txjh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh","openshift-monitoring/thanos-querier-5c8658f985-v8kgg","kube-system/konnectivity-agent-zllgp","openshift-ingress-canary/ingress-canary-j22f2","openshift-image-registry/node-ca-kxg89","kube-system/global-pull-secret-syncer-plzrc","kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal","openshift-monitoring/node-exporter-w8v99","openshift-monitoring/prometheus-k8s-0","openshift-dns/node-resolver-c96l5"] Oct 30 18:37:57.169868 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.169838 5869 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:57.169967 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:57.169882 5869 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:57.172320 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.172306 5869 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:57.172369 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:57.172337 5869 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.us-west-2.compute.internal" Oct 30 18:37:57.199646 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.199610 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kp722\"" Oct 30 18:37:57.199722 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.199658 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:37:57.200042 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.200024 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.200307 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.200284 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.200368 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.200354 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k6xhn\"" Oct 30 18:37:57.207188 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.207166 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:37:57.207308 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.207290 5869 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:37:57.207885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.207864 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vmn22\"" Oct 30 18:37:57.209607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.209592 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-h66ck\"" Oct 30 18:37:57.211859 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.211844 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w6mkd\"" Oct 30 18:37:57.213591 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213575 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-p85r8\"" Oct 30 18:37:57.213591 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213587 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-kxkbv\"" Oct 30 18:37:57.213794 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213775 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5fxhm\"" Oct 30 18:37:57.213849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213834 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-thsk7\"" Oct 30 18:37:57.213892 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213873 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:37:57.213979 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.213960 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-mrt5r\"" Oct 30 18:37:57.214051 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214036 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:37:57.214173 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214157 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:37:57.214247 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214197 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-djndp\"" Oct 30 18:37:57.214247 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214200 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:37:57.214247 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214213 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sslwz\"" Oct 30 18:37:57.214247 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214245 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j7txc\"" Oct 30 18:37:57.214406 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.214297 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k96h\"" Oct 30 18:37:57.215628 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.215614 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:37:57.216109 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216081 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216183 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216135 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216372 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216350 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216377 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216425 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216454 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216475 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216486 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:37:57.216492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216425 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216511 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216541 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:37:57.216748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216556 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216694 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216836 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216843 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.216915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216839 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.216915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216887 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.217018 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216955 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:37:57.217018 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.216988 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:37:57.217018 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217007 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:37:57.217018 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217011 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.217230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217020 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.217230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217085 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:37:57.217230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217119 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:37:57.217230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217162 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:37:57.217230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217221 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.217403 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217248 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:37:57.217403 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.217285 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:37:57.220533 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.220520 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.223528 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.223512 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:37:57.223616 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.223601 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:37:57.223668 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.223637 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q9k2d\"" Oct 30 18:37:57.223716 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.223665 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:37:57.223716 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.223696 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:37:57.232268 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.232252 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hd9r2\"" Oct 30 18:37:57.288693 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288633 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.288693 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288667 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.288693 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288687 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.288860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288714 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.288860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288796 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-tmp\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.288860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288837 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.288998 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288876 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.288998 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288908 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.288998 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288937 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.288998 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288963 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.288996 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289044 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289052 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289133 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289176 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpjb\" (UniqueName: \"kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289209 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.289246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289241 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289270 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289286 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-metrics-tls\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289295 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289383 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e70452c4-c454-4edd-857c-3eb8d36acd12-cert\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289393 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289429 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289447 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289470 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0e02e4a-2151-4339-93de-09057afd8f9d-iptables-alerter-script\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289485 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289502 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6cc6\" (UniqueName: \"kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289518 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.289544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289534 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289554 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289573 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289590 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab46616b-1670-40b4-97be-689b9931022c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289608 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-env-overrides\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289628 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289676 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289680 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289724 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289751 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289839 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289873 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289884 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-agent-certs\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:37:57.289970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289934 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289997 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.289994 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab46616b-1670-40b4-97be-689b9931022c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290050 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2dc\" (UniqueName: \"kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290086 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290124 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/182b13f1-6909-4251-8455-eed4d52ae204-original-pull-secret\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290145 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290166 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290172 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-config\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290187 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6sh\" (UniqueName: \"kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290208 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290231 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290270 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7268393d-9be4-4e1c-87ff-c9ac7cfd7072-konnectivity-ca\") pod \"konnectivity-agent-zllgp\" (UID: \"7268393d-9be4-4e1c-87ff-c9ac7cfd7072\") " pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290277 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290299 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290339 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290382 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98jlw\" (UniqueName: \"kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290426 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.290447 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290445 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290481 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290509 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290539 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290557 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovnkube-script-lib\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290564 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290590 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290615 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290645 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290729 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48fqm\" (UniqueName: \"kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290759 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpzt\" (UniqueName: \"kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290789 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290810 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/166168ae-19c0-4650-9a99-be2f63f2917f-metrics-certs\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290815 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290837 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-config-volume\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290901 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290932 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv87d\" (UniqueName: \"kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290950 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290965 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.290993 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.291076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291026 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291085 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291144 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291172 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05a02e6e-236c-4782-95af-7e99fa284781-serviceca\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291176 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291236 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291286 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291320 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291345 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291393 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291435 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291470 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cll7\" (UniqueName: \"kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291492 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291542 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291592 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291632 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291678 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291707 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291734 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.291761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291762 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291795 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdde0beb-f496-4013-aff3-3706e199e7f2-tmp\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291800 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291831 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfk42\" (UniqueName: \"kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291858 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291904 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291943 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291963 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-cni-binary-copy\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291967 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.291993 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292019 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292050 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292106 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292138 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v562v\" (UniqueName: \"kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292164 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292182 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292207 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292265 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-audit-log\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292288 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292328 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rd6\" (UniqueName: \"kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292373 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292397 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292418 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sg2c\" (UniqueName: \"kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292437 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292453 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjk5\" (UniqueName: \"kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292464 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/79e792e9-28c1-4127-b364-3679fcf27c7c-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.292458 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292470 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292508 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292528 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292545 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292568 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292594 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292622 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292632 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292682 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292719 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292746 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292777 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwsp\" (UniqueName: \"kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292807 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292843 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292858 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292912 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.292985 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvck9\" (UniqueName: \"kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293004 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293020 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293037 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293068 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293192 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293214 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293271 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-textfile\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293282 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.293329 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293320 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293349 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293380 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg47z\" (UniqueName: \"kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293395 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-daemon-config\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293426 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293445 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293462 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293486 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293504 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293558 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293576 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293598 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293614 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293629 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293648 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293677 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293703 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293711 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab46616b-1670-40b4-97be-689b9931022c-data-volume\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293724 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-tmp-dir\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293778 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293813 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293842 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293869 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293888 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-tuned\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293895 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293924 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293950 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.293975 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294004 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294023 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-tmp-dir\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294031 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294126 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294151 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294173 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g84q\" (UniqueName: \"kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:57.294197 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294197 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294254 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294279 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294364 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294387 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294429 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294462 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294504 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294522 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294549 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294570 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294589 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.295291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.294691 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52ca3b9e-8619-467a-a705-5bcbabdd7a08-ovn-node-metrics-cert\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.295699 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295442 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 18:37:57.295699 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295501 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:37:57.295699 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295590 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 18:37:57.295699 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295693 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:37:57.295802 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295724 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:37:57.295881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.295866 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:37:57.296157 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296082 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:37:57.296157 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296109 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-h6fl7\"" Oct 30 18:37:57.296260 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296157 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:37:57.296306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296295 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:37:57.296355 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296315 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:37:57.296355 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296322 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296529 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cj5hm9jbahbsc\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296537 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296551 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296539 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296549 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296587 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6qrp7\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296598 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296603 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-35h58a6i46adv\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296606 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lr79t\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296606 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296638 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296633 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296640 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296640 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296650 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:37:57.296691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296687 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9rjdp\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296783 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296805 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5l64w\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296817 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296823 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296828 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r7d57\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296805 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-acesbpb1nq1nl\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296857 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296859 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296860 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296882 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296885 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:37:57.297296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.296927 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tj5qj\"" Oct 30 18:37:57.299240 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.299173 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.299240 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.299185 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.299240 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.299215 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.299395 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.299296 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.299395 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.299303 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.300388 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300366 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.300388 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300377 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.300534 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300409 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.300534 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300385 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.300534 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300458 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.300670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300533 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-client-certs\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.300670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300539 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef44a6cf-b792-4c9c-8503-271d365976f8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:57.300670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300584 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:37:57.300670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300593 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 18:37:57.300670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300655 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:37:57.300865 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300675 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:37:57.300865 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300709 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.300865 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300834 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.300965 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.300899 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:37:57.301081 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301066 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.301243 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301225 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-secret-metrics-server-tls\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.301298 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301255 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 18:37:57.301298 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301279 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:37:57.301434 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301419 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.301668 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301591 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-grpc-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.301737 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301642 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 18:37:57.301795 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301774 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:37:57.302023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301994 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a6523be-9435-46c7-b5a2-b84a2bba0d83-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.302201 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.301998 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.302503 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.302483 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.303046 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.302997 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.303154 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303058 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.303154 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303134 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-metrics-server-audit-profiles\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.303286 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303177 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.303286 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303178 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bdde0beb-f496-4013-aff3-3706e199e7f2-metrics-client-ca\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.303386 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303338 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.303386 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303376 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.303542 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303525 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.303619 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303601 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.303686 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303668 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.303752 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303738 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.303785 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303762 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-client-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.303907 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303894 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6523be-9435-46c7-b5a2-b84a2bba0d83-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.303938 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303894 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.303999 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.303983 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-metrics-client-ca\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.304161 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.304148 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.304331 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.304293 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.304331 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.304322 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.304405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.304324 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bdde0beb-f496-4013-aff3-3706e199e7f2-secret-thanos-querier-tls\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.304434 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.304418 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-tls\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.305567 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.305549 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.305637 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.305623 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.308857 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.308841 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.309994 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.309979 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.311159 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.311145 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e792e9-28c1-4127-b364-3679fcf27c7c-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.311204 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.311145 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.344019 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.343985 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:37:57.344504 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.344486 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:37:57.344581 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.344500 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:37:57.344764 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.344745 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.346627 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.346596 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98jlw\" (UniqueName: \"kubernetes.io/projected/7f5e5aef-db22-432b-9bfc-64231443a33c-kube-api-access-98jlw\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.346757 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.346734 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6cc6\" (UniqueName: \"kubernetes.io/projected/e70452c4-c454-4edd-857c-3eb8d36acd12-kube-api-access-w6cc6\") pod \"ingress-canary-j22f2\" (UID: \"e70452c4-c454-4edd-857c-3eb8d36acd12\") " pod="openshift-ingress-canary/ingress-canary-j22f2" Oct 30 18:37:57.347159 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.347119 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2dc\" (UniqueName: \"kubernetes.io/projected/734c247a-5ffb-4ccc-9720-8ba9d5ac3711-kube-api-access-kd2dc\") pod \"metrics-server-56686866c6-xhcgv\" (UID: \"734c247a-5ffb-4ccc-9720-8ba9d5ac3711\") " pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:37:57.347252 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.347204 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6sh\" (UniqueName: \"kubernetes.io/projected/ab46616b-1670-40b4-97be-689b9931022c-kube-api-access-qh6sh\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.347574 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.347556 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpjb\" (UniqueName: \"kubernetes.io/projected/05a02e6e-236c-4782-95af-7e99fa284781-kube-api-access-fjpjb\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.348561 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.348537 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.348946 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.348923 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpzt\" (UniqueName: \"kubernetes.io/projected/7a6523be-9435-46c7-b5a2-b84a2bba0d83-kube-api-access-5qpzt\") pod \"openshift-state-metrics-7bd7489957-hwjnm\" (UID: \"7a6523be-9435-46c7-b5a2-b84a2bba0d83\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-hwjnm" Oct 30 18:37:57.349145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.349127 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv87d\" (UniqueName: \"kubernetes.io/projected/e4e82816-cfc3-413b-a254-47dd3a2535ea-kube-api-access-tv87d\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.349323 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.349307 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"alertmanager-main-0\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:37:57.349364 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.349336 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48fqm\" (UniqueName: \"kubernetes.io/projected/166168ae-19c0-4650-9a99-be2f63f2917f-kube-api-access-48fqm\") pod \"network-metrics-daemon-sndlp\" (UID: \"166168ae-19c0-4650-9a99-be2f63f2917f\") " pod="openshift-multus/network-metrics-daemon-sndlp" Oct 30 18:37:57.349498 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.349480 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cll7\" (UniqueName: \"kubernetes.io/projected/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-kube-api-access-2cll7\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.352491 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.352473 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.365230 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.365207 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfk42\" (UniqueName: \"kubernetes.io/projected/3740ec35-b3ca-4b78-a3f2-610eeb7b3ada-kube-api-access-jfk42\") pod \"dns-default-7txjh\" (UID: \"3740ec35-b3ca-4b78-a3f2-610eeb7b3ada\") " pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:57.367202 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.366776 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwsp\" (UniqueName: \"kubernetes.io/projected/f4cc7620-49dc-4d5a-8343-411e0d17309d-kube-api-access-sgwsp\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.367202 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.366808 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v562v\" (UniqueName: \"kubernetes.io/projected/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-kube-api-access-v562v\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.367906 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6rd6\" (UniqueName: \"kubernetes.io/projected/8bf1866b-1e63-4886-830d-c8645a8e5ff5-kube-api-access-n6rd6\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.368406 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvck9\" (UniqueName: \"kubernetes.io/projected/52ca3b9e-8619-467a-a705-5bcbabdd7a08-kube-api-access-kvck9\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369004 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"prometheus-k8s-0\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369122 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sg2c\" (UniqueName: \"kubernetes.io/projected/79e792e9-28c1-4127-b364-3679fcf27c7c-kube-api-access-7sg2c\") pod \"kube-state-metrics-5dc8d7d89b-qc4zv\" (UID: \"79e792e9-28c1-4127-b364-3679fcf27c7c\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-qc4zv" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369607 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9mm\" (UniqueName: \"kubernetes.io/projected/7190032a-e51d-4bd0-95e5-30f6f95e59a4-kube-api-access-9h9mm\") pod \"network-check-target-wnj4b\" (UID: \"7190032a-e51d-4bd0-95e5-30f6f95e59a4\") " pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369806 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g84q\" (UniqueName: \"kubernetes.io/projected/ef44a6cf-b792-4c9c-8503-271d365976f8-kube-api-access-4g84q\") pod \"monitoring-plugin-77f4f76594-8lpn2\" (UID: \"ef44a6cf-b792-4c9c-8503-271d365976f8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369860 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg47z\" (UniqueName: \"kubernetes.io/projected/f0e02e4a-2151-4339-93de-09057afd8f9d-kube-api-access-lg47z\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.370712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.369979 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjk5\" (UniqueName: \"kubernetes.io/projected/bdde0beb-f496-4013-aff3-3706e199e7f2-kube-api-access-lgjk5\") pod \"thanos-querier-5c8658f985-v8kgg\" (UID: \"bdde0beb-f496-4013-aff3-3706e199e7f2\") " pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:37:57.394903 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394882 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.394988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394909 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.394988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394925 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.394988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394940 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.394988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394955 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.394988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.394976 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395001 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395008 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-multus\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395015 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395025 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395036 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395042 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-systemd-units\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395042 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-etc-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395066 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395072 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-system-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395112 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-log-socket\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395125 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-device-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395139 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-k8s-cni-cncf-io\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395138 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395178 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395180 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-socket-dir-parent\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395199 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395205 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395225 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395236 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-systemd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395241 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395239 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395268 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395281 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-netns\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395286 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395318 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-dbus\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395321 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-modprobe-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395329 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7-hosts-file\") pod \"node-resolver-c96l5\" (UID: \"ef3aa381-aaa3-4f19-9df0-dbc93b3b1fe7\") " pod="openshift-dns/node-resolver-c96l5" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395332 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395358 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395385 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395406 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395411 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-var-lib-kubelet\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395426 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-kubelet\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395435 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-ovn\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395434 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395464 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395469 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05a02e6e-236c-4782-95af-7e99fa284781-host\") pod \"node-ca-kxg89\" (UID: \"05a02e6e-236c-4782-95af-7e99fa284781\") " pod="openshift-image-registry/node-ca-kxg89" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395473 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395486 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395507 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395520 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395543 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395557 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-bin\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395558 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-kubernetes\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395520 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-run\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395572 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395579 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-cnibin\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395589 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395603 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395618 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395631 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395603 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bf1866b-1e63-4886-830d-c8645a8e5ff5-system-cni-dir\") pod \"multus-additional-cni-plugins-jnv56\" (UID: \"8bf1866b-1e63-4886-830d-c8645a8e5ff5\") " pod="openshift-multus/multus-additional-cni-plugins-jnv56" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395643 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-run-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395644 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395666 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-hostroot\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395669 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-slash\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395672 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395687 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395690 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-sys\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395699 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395717 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-systemd\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395728 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395744 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/182b13f1-6909-4251-8455-eed4d52ae204-kubelet-config\") pod \"global-pull-secret-syncer-plzrc\" (UID: \"182b13f1-6909-4251-8455-eed4d52ae204\") " pod="kube-system/global-pull-secret-syncer-plzrc" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395747 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e02e4a-2151-4339-93de-09057afd8f9d-host-slash\") pod \"iptables-alerter-rpdb6\" (UID: \"f0e02e4a-2151-4339-93de-09057afd8f9d\") " pod="openshift-network-operator/iptables-alerter-rpdb6" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395752 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395757 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-host\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395775 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395781 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395784 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-root\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395799 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395819 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-etc-kubernetes\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395823 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395800 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-var-lib-openvswitch\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395839 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysconfig\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395845 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395864 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-kubelet\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395871 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.395860 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395893 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395914 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395928 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395946 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-netns\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395945 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395872 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab46616b-1670-40b4-97be-689b9931022c-crio-socket\") pod \"insights-runtime-extractor-rd622\" (UID: \"ab46616b-1670-40b4-97be-689b9931022c\") " pod="openshift-insights/insights-runtime-extractor-rd622" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395970 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395992 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.395999 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-lib-modules\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396001 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-var-lib-cni-bin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396017 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396021 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-cnibin\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396029 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396039 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396047 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-conf\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396053 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1df3f31-27af-4f32-aa1e-e07cf031f2cf-node-exporter-wtmp\") pod \"node-exporter-w8v99\" (UID: \"a1df3f31-27af-4f32-aa1e-e07cf031f2cf\") " pod="openshift-monitoring/node-exporter-w8v99" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396058 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396088 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-cni-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396118 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4e82816-cfc3-413b-a254-47dd3a2535ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8hrdh\" (UID: \"e4e82816-cfc3-413b-a254-47dd3a2535ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8hrdh" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396137 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-etc-sysctl-d\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396152 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396167 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396182 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396196 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396181 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-cni-netd\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396210 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396210 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-host-run-multus-certs\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396227 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396239 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52ca3b9e-8619-467a-a705-5bcbabdd7a08-node-log\") pod \"ovnkube-node-dbz7h\" (UID: \"52ca3b9e-8619-467a-a705-5bcbabdd7a08\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396242 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4cc7620-49dc-4d5a-8343-411e0d17309d-multus-conf-dir\") pod \"multus-25pf7\" (UID: \"f4cc7620-49dc-4d5a-8343-411e0d17309d\") " pod="openshift-multus/multus-25pf7" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396270 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.397679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.396334 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f5e5aef-db22-432b-9bfc-64231443a33c-sys\") pod \"tuned-4xjt2\" (UID: \"7f5e5aef-db22-432b-9bfc-64231443a33c\") " pod="openshift-cluster-node-tuning-operator/tuned-4xjt2" Oct 30 18:37:57.716672 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:57.716633 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7txjh" Oct 30 18:37:58.607827 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:58.607802 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:37:59.091189 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:37:59.091163 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:37:59.279570 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:37:59.279536 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:38:00.039938 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:00.039906 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c8658f985-v8kgg" Oct 30 18:38:00.088156 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:00.088131 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:38:00.088378 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:00.088362 5869 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:38:00.371714 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:00.371648 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-8lpn2" Oct 30 18:38:04.490524 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:04.490491 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zllgp" Oct 30 18:38:06.610809 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:06.610739 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56686866c6-xhcgv" Oct 30 18:38:06.880750 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:06.880689 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:38:09.914254 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:09.914225 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbz7h" Oct 30 18:38:17.156371 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156333 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:17.156933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156756 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="alertmanager" containerID="cri-o://3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" gracePeriod=120 Oct 30 18:38:17.156933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156799 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy" containerID="cri-o://695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" gracePeriod=120 Oct 30 18:38:17.156933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156847 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="prom-label-proxy" containerID="cri-o://2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" gracePeriod=120 Oct 30 18:38:17.156933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156851 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-web" containerID="cri-o://a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" gracePeriod=120 Oct 30 18:38:17.156933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156899 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-metric" containerID="cri-o://6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" gracePeriod=120 Oct 30 18:38:17.157209 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.156971 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="config-reloader" containerID="cri-o://5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" gracePeriod=120 Oct 30 18:38:17.437739 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.437443 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:17.526487 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526462 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526645 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526495 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526645 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526522 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526645 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526548 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526815 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526655 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526815 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526709 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526815 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526747 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526816 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526849 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526879 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526896 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526914 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.526964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526952 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.527271 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.526986 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") pod \"49cb5f50-428f-433b-b372-5b6e19584741\" (UID: \"49cb5f50-428f-433b-b372-5b6e19584741\") " Oct 30 18:38:17.527271 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.527259 5869 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-metrics-client-ca\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.527877 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.527573 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:17.528351 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.527995 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:17.530827 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.529899 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.530827 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.530001 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.530827 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.530206 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.531537 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.531510 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:17.531665 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.531636 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.531984 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.531958 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "kube-api-access-x8flf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:17.532140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.532088 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:17.532140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.532131 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.535673 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.535650 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.542396 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.542373 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config" pod "49cb5f50-428f-433b-b372-5b6e19584741" (UID: "49cb5f50-428f-433b-b372-5b6e19584741"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:17.627830 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627789 5869 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-main-db\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.627830 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627823 5869 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-tls-assets\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627838 5869 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-cluster-tls-config\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627853 5869 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-config-volume\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627868 5869 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627882 5869 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49cb5f50-428f-433b-b372-5b6e19584741-config-out\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627897 5869 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627913 5869 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-web-config\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627927 5869 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-main-tls\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627942 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8flf\" (UniqueName: \"kubernetes.io/projected/49cb5f50-428f-433b-b372-5b6e19584741-kube-api-access-x8flf\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627957 5869 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49cb5f50-428f-433b-b372-5b6e19584741-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:17.628021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:17.627972 5869 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49cb5f50-428f-433b-b372-5b6e19584741-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:18.184421 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184394 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" exitCode=0 Oct 30 18:38:18.184421 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184417 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" exitCode=0 Oct 30 18:38:18.184421 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184424 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" exitCode=0 Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184430 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" exitCode=0 Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184436 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" exitCode=0 Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184442 5869 generic.go:356] "Generic (PLEG): container finished" podID="49cb5f50-428f-433b-b372-5b6e19584741" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" exitCode=0 Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184504 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184564 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184567 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184591 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184617 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184633 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184642 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184667 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} Oct 30 18:38:18.184943 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.184700 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"49cb5f50-428f-433b-b372-5b6e19584741","Type":"ContainerDied","Data":"d485655269704b2e25971fa07be6b0de7957cbf092d7bc1e04b3b8a9f03da137"} Oct 30 18:38:18.192487 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.192468 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.199493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.199476 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.206183 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.206164 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.213078 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.213061 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.219712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.219690 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.226371 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.226355 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.233150 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.233135 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.233452 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.233417 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.233502 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.233454 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.233502 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.233493 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.233776 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.233748 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.233870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.233774 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.233870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.233795 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.234011 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.233993 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.234059 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234016 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.234059 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234029 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.234258 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.234243 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.234295 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234261 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.234295 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234272 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.234511 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.234487 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.234583 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234518 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.234583 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234538 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.234806 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.234788 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.234852 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234814 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.234852 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.234831 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.235074 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:18.235056 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.235144 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235078 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.235144 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235109 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.235369 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235351 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.235427 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235371 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.235607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235589 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.235671 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235608 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.235857 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235840 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.235899 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.235858 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.236056 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236035 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.236056 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236058 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.236267 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236243 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.236306 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236268 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.236476 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236460 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.236520 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236478 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.236675 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236657 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.236725 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236675 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.236893 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236874 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.236937 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.236894 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.237149 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237130 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.237149 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237146 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.237359 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237344 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.237405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237360 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.237596 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237577 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.237638 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237597 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.237876 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237846 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.237918 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.237876 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.238133 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238085 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.238186 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238136 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.238392 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238376 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.238443 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238393 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.238617 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238597 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.238694 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238618 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.238874 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238856 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.238918 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.238875 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.239159 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239138 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.239234 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239159 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.239582 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239558 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.239582 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239582 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.239933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239847 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.239933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.239871 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.240485 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.240462 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.240485 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.240483 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.240758 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.240740 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.240813 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.240758 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.241019 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.240998 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.241019 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241019 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.241307 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241267 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.241382 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241308 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.241570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241547 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.241570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241565 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.241840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241820 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.241887 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.241840 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.242110 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242072 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.242164 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242109 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.242359 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242342 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.242403 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242359 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.242575 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242560 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.242614 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242575 5869 scope.go:122] "RemoveContainer" containerID="2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569" Oct 30 18:38:18.242799 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242784 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569"} err="failed to get container status \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": rpc error: code = NotFound desc = could not find container \"2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569\": container with ID starting with 2de56115121de6803b49114fc14bca8a3f702f3e40e42f063b4f5767ac319569 not found: ID does not exist" Oct 30 18:38:18.242799 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242798 5869 scope.go:122] "RemoveContainer" containerID="6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6" Oct 30 18:38:18.242997 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242983 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6"} err="failed to get container status \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": rpc error: code = NotFound desc = could not find container \"6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6\": container with ID starting with 6e4b4d96ad8c42679cb069b1657a1e5bfaf0b155220f54593ad300d7c082d8a6 not found: ID does not exist" Oct 30 18:38:18.243038 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.242997 5869 scope.go:122] "RemoveContainer" containerID="695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86" Oct 30 18:38:18.243219 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243201 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86"} err="failed to get container status \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": rpc error: code = NotFound desc = could not find container \"695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86\": container with ID starting with 695c7e28f4578de9e2fbaa84d8122dc2eb25183a108c57e788514d6eaebe3c86 not found: ID does not exist" Oct 30 18:38:18.243273 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243220 5869 scope.go:122] "RemoveContainer" containerID="a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87" Oct 30 18:38:18.243435 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243421 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87"} err="failed to get container status \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": rpc error: code = NotFound desc = could not find container \"a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87\": container with ID starting with a1e5895e07ceedc6930b4b941031c5db0a281c6698f495d8d8a066720a75ed87 not found: ID does not exist" Oct 30 18:38:18.243482 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243435 5869 scope.go:122] "RemoveContainer" containerID="5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5" Oct 30 18:38:18.243650 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243632 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5"} err="failed to get container status \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": rpc error: code = NotFound desc = could not find container \"5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5\": container with ID starting with 5eee748c73e5d64acee69e5c3765d2675232cf6feb8aad505e54d69baa5acae5 not found: ID does not exist" Oct 30 18:38:18.243715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243651 5869 scope.go:122] "RemoveContainer" containerID="3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62" Oct 30 18:38:18.243893 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243878 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62"} err="failed to get container status \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": rpc error: code = NotFound desc = could not find container \"3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62\": container with ID starting with 3d07e3aa474abf92d61f724b8e2ab712c29e0f2fd91a134cfa9efb28df713e62 not found: ID does not exist" Oct 30 18:38:18.243940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.243894 5869 scope.go:122] "RemoveContainer" containerID="ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403" Oct 30 18:38:18.244120 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.244087 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403"} err="failed to get container status \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": rpc error: code = NotFound desc = could not find container \"ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403\": container with ID starting with ad0673ef92d914f77fd2c6abe4c134ce40923ea01efd36f46fdcde097c0ff403 not found: ID does not exist" Oct 30 18:38:18.363964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.363907 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:18.411428 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.411405 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:18.437805 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.437781 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:18.438051 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438039 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="config-reloader" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438053 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="config-reloader" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438068 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-web" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438073 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-web" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438084 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438088 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438120 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="init-config-reloader" Oct 30 18:38:18.438123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438124 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="init-config-reloader" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438132 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="alertmanager" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438137 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="alertmanager" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438146 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="prom-label-proxy" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438151 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="prom-label-proxy" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438158 5869 cpu_manager.go:414] "containerMap: removing container" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-metric" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438162 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-metric" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438200 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="prom-label-proxy" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438207 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-metric" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438214 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="config-reloader" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438220 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy-web" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438226 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="kube-rbac-proxy" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438232 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="init-config-reloader" Oct 30 18:38:18.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.438237 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="49cb5f50-428f-433b-b372-5b6e19584741" containerName="alertmanager" Oct 30 18:38:18.471517 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.471498 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.522326 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.522304 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:18.535250 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535219 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npwgj\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-kube-api-access-npwgj\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535347 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535271 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-out\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535398 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535347 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-web-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535398 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535388 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535493 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535418 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535567 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535546 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535636 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535620 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535704 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535684 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535754 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535723 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535809 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535794 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535849 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535831 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535859 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.535934 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.535886 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.569055 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569024 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569236 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569269 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569273 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569297 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569300 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lr79t\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569300 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569326 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569319 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:38:18.569342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.569346 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:38:18.641030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.640966 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npwgj\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-kube-api-access-npwgj\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641002 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-out\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641018 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-web-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641052 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641075 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641125 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641184 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641215 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641235 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641267 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641289 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641661 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641315 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641661 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641345 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.641661 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641387 5869 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:38:18.641661 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.641496 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.642046 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.642024 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.642529 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.642505 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.645020 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.645000 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-out\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.646927 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.646900 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647033 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.646929 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647033 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.646957 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647325 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.647299 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647420 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.647327 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647544 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.647521 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.647659 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.647640 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-web-config\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.648840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.648821 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.699328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.699297 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npwgj\" (UniqueName: \"kubernetes.io/projected/a693e16c-af01-4ddd-b008-a1cf5cf99a4d-kube-api-access-npwgj\") pod \"alertmanager-main-0\" (UID: \"a693e16c-af01-4ddd-b008-a1cf5cf99a4d\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:18.873224 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:18.873201 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:38:19.378038 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:19.378011 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:38:19.405340 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:38:19.405315 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda693e16c_af01_4ddd_b008_a1cf5cf99a4d.slice/crio-f4c67db59ae3e541562d74d2db5839544ad157725b7d01cd1f3b6d278892aa80 WatchSource:0}: Error finding container f4c67db59ae3e541562d74d2db5839544ad157725b7d01cd1f3b6d278892aa80: Status 404 returned error can't find the container with id f4c67db59ae3e541562d74d2db5839544ad157725b7d01cd1f3b6d278892aa80 Oct 30 18:38:20.040852 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:20.040818 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49cb5f50-428f-433b-b372-5b6e19584741" path="/var/lib/kubelet/pods/49cb5f50-428f-433b-b372-5b6e19584741/volumes" Oct 30 18:38:20.193435 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:20.193405 5869 generic.go:356] "Generic (PLEG): container finished" podID="a693e16c-af01-4ddd-b008-a1cf5cf99a4d" containerID="eebb42b0c73524c0ff9a40acc2da2ff35479ee413913731276c0d25f08ff2289" exitCode=0 Oct 30 18:38:20.193557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:20.193471 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerDied","Data":"eebb42b0c73524c0ff9a40acc2da2ff35479ee413913731276c0d25f08ff2289"} Oct 30 18:38:20.193557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:20.193528 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"f4c67db59ae3e541562d74d2db5839544ad157725b7d01cd1f3b6d278892aa80"} Oct 30 18:38:21.190844 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.190809 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bdcd88dbb-94dqj"] Oct 30 18:38:21.193446 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.193425 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202297 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"72a175e0f528f84a28c5ca8569146dea4e9d1fc609e5c5d0a1a8968d333a6ed5"} Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202421 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"db31890f69d33a16bad93c8455951882911bb346991c41b37dfe6280cef9e240"} Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202480 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"7858c8dc25d99d39fe548e0830a00877da5bd0e0a8c0d37f83e80e40da1bfcba"} Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202530 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"cd1562b77762196fa50779118ceeffdc7136d791d8c81040da52a6a6cdbde9d1"} Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202579 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"0bb9a3f019bfbd50701966ed07dcd676e91b0dbadd95247414a7e8545ef87aba"} Oct 30 18:38:21.206235 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.202628 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a693e16c-af01-4ddd-b008-a1cf5cf99a4d","Type":"ContainerStarted","Data":"67a9b66b2ed6190b1b1345d0ce3d46eb5a89379cfe493ccd4407bf5038edcb1a"} Oct 30 18:38:21.254356 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254327 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 18:38:21.254650 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254624 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 18:38:21.254650 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254632 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 18:38:21.254840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254638 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k8qsg\"" Oct 30 18:38:21.254840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254682 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 18:38:21.254840 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.254646 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 18:38:21.262495 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.262467 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.262942 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.262920 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-federate-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.262959 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-serving-certs-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.262978 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw6vc\" (UniqueName: \"kubernetes.io/projected/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-kube-api-access-vw6vc\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.263008 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.263022 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263263 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.263088 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.263321 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.263261 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-metrics-client-ca\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.302258 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.302232 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 18:38:21.303045 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.303026 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bdcd88dbb-94dqj"] Oct 30 18:38:21.364118 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364075 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364228 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364132 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364228 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364169 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364228 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364221 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-metrics-client-ca\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364396 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364269 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364396 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364316 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-federate-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364396 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364343 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-serving-certs-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.364396 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.364370 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw6vc\" (UniqueName: \"kubernetes.io/projected/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-kube-api-access-vw6vc\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.365048 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.365020 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-serving-certs-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.365165 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.365032 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.365265 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.365240 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-metrics-client-ca\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.366723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.366696 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.366856 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.366839 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-secret-telemeter-client\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.366918 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.366881 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-telemeter-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.367201 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.367177 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-federate-client-tls\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.426844 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.426825 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw6vc\" (UniqueName: \"kubernetes.io/projected/69ca6d2f-16b4-4af4-be21-0dfec25fdcbb-kube-api-access-vw6vc\") pod \"telemeter-client-bdcd88dbb-94dqj\" (UID: \"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb\") " pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.441342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.441267 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.428003611 podStartE2EDuration="3.441252126s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.248515ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:21.440466482 +0000 UTC m=+25.512290558" watchObservedRunningTime="2026-10-30 18:38:21.441252126 +0000 UTC m=+25.513076204" Oct 30 18:38:21.512668 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.512635 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" Oct 30 18:38:21.928066 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.928045 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bdcd88dbb-94dqj"] Oct 30 18:38:21.930171 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:38:21.930142 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ca6d2f_16b4_4af4_be21_0dfec25fdcbb.slice/crio-8ee3f8e24acef30645b1c5535e11252a327360e591091addf242748451f9500c WatchSource:0}: Error finding container 8ee3f8e24acef30645b1c5535e11252a327360e591091addf242748451f9500c: Status 404 returned error can't find the container with id 8ee3f8e24acef30645b1c5535e11252a327360e591091addf242748451f9500c Oct 30 18:38:21.932310 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:21.932291 5869 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:38:22.208389 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:22.208360 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" event={"ID":"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb","Type":"ContainerStarted","Data":"8ee3f8e24acef30645b1c5535e11252a327360e591091addf242748451f9500c"} Oct 30 18:38:23.320519 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.320483 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:23.320915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.320891 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy" containerID="cri-o://e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" gracePeriod=600 Oct 30 18:38:23.320964 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.320876 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="prometheus" containerID="cri-o://43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" gracePeriod=600 Oct 30 18:38:23.321183 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.321154 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-thanos" containerID="cri-o://38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" gracePeriod=600 Oct 30 18:38:23.321299 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.321278 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="config-reloader" containerID="cri-o://5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" gracePeriod=600 Oct 30 18:38:23.321570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.321351 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-web" containerID="cri-o://2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" gracePeriod=600 Oct 30 18:38:23.321570 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.321383 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="thanos-sidecar" containerID="cri-o://b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" gracePeriod=600 Oct 30 18:38:23.605406 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.605376 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:23.682123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682071 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682266 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682138 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682266 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682180 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682266 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682225 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682266 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682255 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682306 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682334 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682373 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682404 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682436 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682467 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682521 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682512 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682552 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682587 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682617 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682652 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682672 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682694 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682719 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.682853 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682755 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d193331c-eb89-4849-b124-76b1bf0d0263\" (UID: \"d193331c-eb89-4849-b124-76b1bf0d0263\") " Oct 30 18:38:23.683253 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.682900 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.683253 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683176 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.683253 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683206 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.683405 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683388 5869 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.683454 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683412 5869 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.683454 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683427 5869 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.683881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683597 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.683881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.683839 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.684434 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.684401 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:23.685042 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.684563 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.685147 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.685083 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "kube-api-access-6xjjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:23.685332 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.685311 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:23.686252 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.686223 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.687918 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.687885 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:23.688584 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.688549 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.688814 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.688793 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.688904 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.688868 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.688967 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.688927 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:23.689018 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.688970 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.689052 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.689031 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.689082 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.689071 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.690431 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.690410 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.698940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.698918 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config" pod "d193331c-eb89-4849-b124-76b1bf0d0263" (UID: "d193331c-eb89-4849-b124-76b1bf0d0263"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:23.784633 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784604 5869 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-config-out\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784633 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784638 5869 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-metrics-client-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784649 5869 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-config\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784658 5869 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784669 5869 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784685 5869 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784707 5869 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-grpc-tls\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784723 5869 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-configmap-metrics-client-ca\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784738 5869 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784753 5869 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-tls-assets\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784767 5869 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784781 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xjjz\" (UniqueName: \"kubernetes.io/projected/d193331c-eb89-4849-b124-76b1bf0d0263-kube-api-access-6xjjz\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784795 5869 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784809 5869 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784822 5869 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-secret-kube-rbac-proxy\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784837 5869 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d193331c-eb89-4849-b124-76b1bf0d0263-prometheus-k8s-db\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:23.784845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:23.784852 5869 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d193331c-eb89-4849-b124-76b1bf0d0263-web-config\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:24.218657 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.218620 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" event={"ID":"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb","Type":"ContainerStarted","Data":"330254480efd52cc247999fd231c6079eddc327288e3edb3f1faadfea9a863d2"} Oct 30 18:38:24.218793 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.218691 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" event={"ID":"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb","Type":"ContainerStarted","Data":"878289c0c230eee9f61f7559fe889c63b8bb9f240d1f065f6aa9ba2f7cd0e3e8"} Oct 30 18:38:24.218793 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.218719 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" event={"ID":"69ca6d2f-16b4-4af4-be21-0dfec25fdcbb","Type":"ContainerStarted","Data":"f126e77e5fa29fb558cbd102950cd18c2d7ee5ea59ab70bf80845a7f36e6060b"} Oct 30 18:38:24.221950 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221926 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" exitCode=0 Oct 30 18:38:24.221950 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221947 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" exitCode=0 Oct 30 18:38:24.221950 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221954 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" exitCode=0 Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221959 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" exitCode=0 Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221965 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" exitCode=0 Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.221970 5869 generic.go:356] "Generic (PLEG): container finished" podID="d193331c-eb89-4849-b124-76b1bf0d0263" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" exitCode=0 Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222008 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222009 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222078 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222133 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.222145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222140 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} Oct 30 18:38:24.222452 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222175 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} Oct 30 18:38:24.222452 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222200 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} Oct 30 18:38:24.222452 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222225 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} Oct 30 18:38:24.222452 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.222255 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d193331c-eb89-4849-b124-76b1bf0d0263","Type":"ContainerDied","Data":"a7db8e500812074ebf9b1b53d42e2900c614dcfcec0d3738ab267118f42ddbfb"} Oct 30 18:38:24.229530 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.229512 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.236392 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.236375 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.242792 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.242774 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.249193 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.249179 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.255719 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.255689 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.262466 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.262447 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.269129 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269114 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.269379 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.269365 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.269428 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269387 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.269428 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269403 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.269636 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.269617 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.269681 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269644 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.269681 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269659 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.269837 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.269820 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.269873 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269840 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.269873 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.269853 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.270064 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.270049 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.270133 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270067 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.270133 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270078 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.270323 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.270309 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.270358 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270325 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.270358 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270336 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.270528 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.270514 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.270562 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270533 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.270562 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270544 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.270732 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:24.270717 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.270770 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270735 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.270770 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270746 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.270977 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270957 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.271030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.270977 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.271198 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271180 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.271248 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271197 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.271455 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271437 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.271490 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271455 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.271666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271648 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.271704 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271667 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.271865 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271849 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.271909 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.271865 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.272106 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272074 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.272165 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272108 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.272410 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272390 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.272410 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272409 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.272647 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272629 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.272689 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272648 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.272848 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272833 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.272848 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.272847 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.273034 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273020 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.273034 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273034 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.273246 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273230 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.273291 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273246 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.273434 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273418 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.273475 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273434 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.273642 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273625 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.273691 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273642 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.273884 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273795 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.273884 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.273807 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.274144 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274118 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.274196 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274146 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.274399 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274376 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.274453 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274399 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.274644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274626 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.274680 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274646 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.274888 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274868 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.274957 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.274890 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.275126 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275084 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.275181 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275129 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.275344 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275327 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.275408 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275346 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.275557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275538 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.275608 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275558 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.275800 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275780 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.275800 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.275799 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.276046 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276028 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.276086 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276047 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.276278 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276260 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.276318 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276281 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.276487 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276470 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.276529 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276487 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.276726 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276707 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.276766 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276726 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.276929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276913 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.276974 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.276929 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.277151 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277135 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.277192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277150 5869 scope.go:122] "RemoveContainer" containerID="38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f" Oct 30 18:38:24.277387 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277365 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f"} err="failed to get container status \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": rpc error: code = NotFound desc = could not find container \"38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f\": container with ID starting with 38a16b660a50d06ff2b2de92af8b63872d76295cc03df72ac630e996e4ba239f not found: ID does not exist" Oct 30 18:38:24.277457 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277389 5869 scope.go:122] "RemoveContainer" containerID="e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934" Oct 30 18:38:24.277627 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277608 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934"} err="failed to get container status \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": rpc error: code = NotFound desc = could not find container \"e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934\": container with ID starting with e4064e6c07e5c436b302ceb473740dceaf442bf545990c5a7283ac64d13f7934 not found: ID does not exist" Oct 30 18:38:24.277671 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277628 5869 scope.go:122] "RemoveContainer" containerID="2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332" Oct 30 18:38:24.277822 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277803 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332"} err="failed to get container status \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": rpc error: code = NotFound desc = could not find container \"2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332\": container with ID starting with 2bb7353c33590443b361c23ee09df1ebd7ab6aa966f35633c90f9853e6d07332 not found: ID does not exist" Oct 30 18:38:24.277822 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.277821 5869 scope.go:122] "RemoveContainer" containerID="b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c" Oct 30 18:38:24.278032 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278016 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c"} err="failed to get container status \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": rpc error: code = NotFound desc = could not find container \"b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c\": container with ID starting with b02da7ed5bbe31071340994d205e797aadc7c4e92cadc15fc4368584eca2fb0c not found: ID does not exist" Oct 30 18:38:24.278070 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278033 5869 scope.go:122] "RemoveContainer" containerID="5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b" Oct 30 18:38:24.278288 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278273 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b"} err="failed to get container status \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": rpc error: code = NotFound desc = could not find container \"5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b\": container with ID starting with 5949aec0d1e468ec12b6af24f14d85505050ecd517741b3270eadd7beca5681b not found: ID does not exist" Oct 30 18:38:24.278393 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278287 5869 scope.go:122] "RemoveContainer" containerID="43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9" Oct 30 18:38:24.278504 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278484 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9"} err="failed to get container status \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": rpc error: code = NotFound desc = could not find container \"43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9\": container with ID starting with 43df4c2eb35ca2fc9629fbf58581e48c0da8722c60a40246aeea2a9819979aa9 not found: ID does not exist" Oct 30 18:38:24.278548 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278505 5869 scope.go:122] "RemoveContainer" containerID="5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb" Oct 30 18:38:24.278715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.278697 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb"} err="failed to get container status \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": rpc error: code = NotFound desc = could not find container \"5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb\": container with ID starting with 5b33f31d8e2f6d03a313a54e8f4a501bfe4010216e0075a8905756db37a7c8cb not found: ID does not exist" Oct 30 18:38:24.356881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.356838 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bdcd88dbb-94dqj" podStartSLOduration=1.300079359 podStartE2EDuration="3.35682832s" totalImagesPullingTime="2.056748961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:24.356623631 +0000 UTC m=+28.428447718" watchObservedRunningTime="2026-10-30 18:38:24.35682832 +0000 UTC m=+28.428652393" Oct 30 18:38:24.469491 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.469468 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:24.476682 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.476626 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:24.548564 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548504 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:24.548832 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548818 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548833 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548841 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="config-reloader" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548846 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="config-reloader" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548853 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="prometheus" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548858 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="prometheus" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548875 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-thanos" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548880 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-thanos" Oct 30 18:38:24.548885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548887 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="init-config-reloader" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548893 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="init-config-reloader" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548900 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="thanos-sidecar" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548904 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="thanos-sidecar" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548913 5869 cpu_manager.go:414] "containerMap: removing container" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-web" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548918 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-web" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548967 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-thanos" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548973 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="init-config-reloader" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548980 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="thanos-sidecar" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548985 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy-web" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548993 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="kube-rbac-proxy" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.548998 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="config-reloader" Oct 30 18:38:24.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.549002 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" containerName="prometheus" Oct 30 18:38:24.552624 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.552607 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590370 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590339 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590370 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590368 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590540 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590389 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590540 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590417 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590540 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590451 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590681 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590545 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590681 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590615 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590782 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590700 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590782 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590731 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590782 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590756 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590883 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590789 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590883 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590829 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.590883 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590859 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590889 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvzz\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-kube-api-access-9qvzz\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590913 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590927 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.590946 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591023 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.591007 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591262 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.591036 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.591262 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.591061 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.605410 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.604980 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:38:24.605540 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.605438 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:38:24.605638 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.605621 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653611 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653645 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653666 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653682 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653716 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653682 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653709 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tj5qj\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653666 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653746 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:38:24.653748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653670 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:38:24.654394 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653715 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-acesbpb1nq1nl\"" Oct 30 18:38:24.654394 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653778 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:38:24.654394 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.653849 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:38:24.654824 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.654804 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:24.691928 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.691905 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.691930 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.691946 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.691961 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.691977 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692000 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692300 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692155 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692300 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692259 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692425 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692464 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692493 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692519 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvzz\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-kube-api-access-9qvzz\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692561 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692583 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692604 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692611 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692767 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692877 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.692954 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692902 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.693547 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.692977 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.693547 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.693007 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.693547 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.693353 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.694270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.694242 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.694400 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.694375 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.695933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.695384 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.695933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.695451 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.695933 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.695891 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.696161 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.696019 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.696491 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.696463 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.697158 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697139 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.697434 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697373 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697629 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697702 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697721 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.697887 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.698143 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698397 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.698365 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.698689 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.698671 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.699178 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.699163 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.768134 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.768088 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvzz\" (UniqueName: \"kubernetes.io/projected/bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6-kube-api-access-9qvzz\") pod \"prometheus-k8s-0\" (UID: \"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:24.880517 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:24.880466 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:25.344562 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:25.344537 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:38:25.346985 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:38:25.346958 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff7919e_2f6b_4c3e_98e4_5195a5c3d2f6.slice/crio-0ae909715e5b8f5c648b041aafb039d76716b25f9a25332a3d9f58553750bbdb WatchSource:0}: Error finding container 0ae909715e5b8f5c648b041aafb039d76716b25f9a25332a3d9f58553750bbdb: Status 404 returned error can't find the container with id 0ae909715e5b8f5c648b041aafb039d76716b25f9a25332a3d9f58553750bbdb Oct 30 18:38:26.039877 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:26.039843 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d193331c-eb89-4849-b124-76b1bf0d0263" path="/var/lib/kubelet/pods/d193331c-eb89-4849-b124-76b1bf0d0263/volumes" Oct 30 18:38:26.231301 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:26.231274 5869 generic.go:356] "Generic (PLEG): container finished" podID="bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6" containerID="406989961d5ca27dedb97cc5cfd99fd29e06552404170482006e036d1cf6db04" exitCode=0 Oct 30 18:38:26.231442 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:26.231330 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerDied","Data":"406989961d5ca27dedb97cc5cfd99fd29e06552404170482006e036d1cf6db04"} Oct 30 18:38:26.231442 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:26.231376 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"0ae909715e5b8f5c648b041aafb039d76716b25f9a25332a3d9f58553750bbdb"} Oct 30 18:38:26.883422 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:26.883395 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wnj4b" Oct 30 18:38:27.238951 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.238918 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"65122aa3d5d3069e28062fc3339bf6e844e638238d72ba05409ac0c764adaf2c"} Oct 30 18:38:27.239345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.238983 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"2d2dc59885e0b5e9bb801d850c6016451b94e1a53b765f6e719f978de8386db0"} Oct 30 18:38:27.239345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.239012 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"1907c6cae8dda3662a656c089748802eac90b1723d5f54fb93d0eebd9407960f"} Oct 30 18:38:27.239345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.239043 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"f8f1b1510e84f982e713b145998ba0e900abf4b3d8aed8faa4ab3b58076ee465"} Oct 30 18:38:27.239345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.239067 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"becb52966bb68c62594169bf49f173773ff55e9e4893566111d55a98bc0f436f"} Oct 30 18:38:27.239345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.239113 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6","Type":"ContainerStarted","Data":"a67b6a93c178640f4df357a1053086df6e3dc71455c368b2c4e2f41e083c613a"} Oct 30 18:38:27.359314 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:27.359265 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.341168356 podStartE2EDuration="3.359252196s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.08384ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:27.358354965 +0000 UTC m=+31.430179055" watchObservedRunningTime="2026-10-30 18:38:27.359252196 +0000 UTC m=+31.431076269" Oct 30 18:38:29.281873 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:29.281827 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:38:29.881520 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:38:29.881481 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:38:59.282411 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:38:59.282362 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:39:25.567887 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:39:25.567845 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:39:25.585347 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:39:25.585321 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:39:26.483393 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:39:26.483369 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:39:29.282190 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:39:29.282151 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:39:59.283614 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:39:59.283565 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:40:29.283516 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:40:29.283475 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:40:59.282249 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:40:59.282205 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:41:29.282158 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:41:29.282055 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:41:59.284007 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:41:59.283967 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:42:29.281975 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:42:29.281936 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:42:59.279891 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:42:59.279855 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:43:29.279870 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:43:29.279820 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:43:59.280677 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:43:59.280631 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:44:19.542746 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.542704 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2"] Oct 30 18:44:19.546416 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.546399 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.560748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.560725 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdv4\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-kube-api-access-cwdv4\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.560862 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.560794 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.599442 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.599422 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7nb4b\"" Oct 30 18:44:19.644974 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.644947 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 18:44:19.645140 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.645011 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 18:44:19.646036 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.646012 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2"] Oct 30 18:44:19.662102 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.662063 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.662201 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.662125 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdv4\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-kube-api-access-cwdv4\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.718577 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.718553 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.719151 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.719132 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdv4\" (UniqueName: \"kubernetes.io/projected/2db6fe37-366c-42d8-8338-5ce91bba9395-kube-api-access-cwdv4\") pod \"cert-manager-cainjector-5c57bbd4fc-xkql2\" (UID: \"2db6fe37-366c-42d8-8338-5ce91bba9395\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:19.870490 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:19.870436 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" Oct 30 18:44:20.233514 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:20.233493 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2"] Oct 30 18:44:20.236183 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:44:20.236158 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2db6fe37_366c_42d8_8338_5ce91bba9395.slice/crio-8cd0439070657f0880c902152cefe1df24b6946356fefcbbdfaa655c3e9bf49c WatchSource:0}: Error finding container 8cd0439070657f0880c902152cefe1df24b6946356fefcbbdfaa655c3e9bf49c: Status 404 returned error can't find the container with id 8cd0439070657f0880c902152cefe1df24b6946356fefcbbdfaa655c3e9bf49c Oct 30 18:44:20.238258 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:20.238239 5869 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:44:20.569880 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:20.569805 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" event={"ID":"2db6fe37-366c-42d8-8338-5ce91bba9395","Type":"ContainerStarted","Data":"8cd0439070657f0880c902152cefe1df24b6946356fefcbbdfaa655c3e9bf49c"} Oct 30 18:44:24.587049 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:24.586968 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" event={"ID":"2db6fe37-366c-42d8-8338-5ce91bba9395","Type":"ContainerStarted","Data":"d7ce20cf09cb5f6e310f67720dca4bcbf57af3145f28dcbd05f65bf300b1c0e2"} Oct 30 18:44:24.704421 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:24.704369 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-xkql2" podStartSLOduration=1.69159266 podStartE2EDuration="5.704354992s" totalImagesPullingTime="4.012762332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:24.704199447 +0000 UTC m=+388.776023522" watchObservedRunningTime="2026-10-30 18:44:24.704354992 +0000 UTC m=+388.776179070" Oct 30 18:44:29.284453 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:44:29.284410 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:44:33.343027 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.342982 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v"] Oct 30 18:44:33.346969 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.346941 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.369160 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.369134 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p26n\" (UniqueName: \"kubernetes.io/projected/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-kube-api-access-2p26n\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.369262 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.369189 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-tmp\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.446985 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.446951 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xz2lk\"" Oct 30 18:44:33.447122 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.446991 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Oct 30 18:44:33.447122 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.446991 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:44:33.447958 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.447934 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v"] Oct 30 18:44:33.470265 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.470241 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-tmp\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.470365 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.470289 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p26n\" (UniqueName: \"kubernetes.io/projected/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-kube-api-access-2p26n\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.470606 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.470586 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-tmp\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.526980 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.526944 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p26n\" (UniqueName: \"kubernetes.io/projected/8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b-kube-api-access-2p26n\") pod \"openshift-lws-operator-8465994787-wgj2v\" (UID: \"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:33.658920 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:33.658873 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" Oct 30 18:44:34.025578 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:34.025422 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v"] Oct 30 18:44:34.028491 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:44:34.028463 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f8fd4ae_e52a_4184_8f19_055ae3e7bc7b.slice/crio-301778e61d038e31038e693776d5d30c57f2fcd1e096a795f2bd850effe84ab7 WatchSource:0}: Error finding container 301778e61d038e31038e693776d5d30c57f2fcd1e096a795f2bd850effe84ab7: Status 404 returned error can't find the container with id 301778e61d038e31038e693776d5d30c57f2fcd1e096a795f2bd850effe84ab7 Oct 30 18:44:34.624885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:34.624847 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" event={"ID":"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b","Type":"ContainerStarted","Data":"301778e61d038e31038e693776d5d30c57f2fcd1e096a795f2bd850effe84ab7"} Oct 30 18:44:37.638944 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:37.638914 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" event={"ID":"8f8fd4ae-e52a-4184-8f19-055ae3e7bc7b","Type":"ContainerStarted","Data":"27951d0b84350ac17c59be5bb67d56079c8992e987115063f375922aca64041f"} Oct 30 18:44:37.751051 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:37.751001 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wgj2v" podStartSLOduration=1.247008686 podStartE2EDuration="4.750988616s" totalImagesPullingTime="3.50397993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:37.750365045 +0000 UTC m=+401.822189123" watchObservedRunningTime="2026-10-30 18:44:37.750988616 +0000 UTC m=+401.822812691" Oct 30 18:44:54.622221 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.622188 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g"] Oct 30 18:44:54.625821 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.625800 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.679138 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.679083 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g"] Oct 30 18:44:54.725475 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.725451 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Oct 30 18:44:54.725597 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.725509 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Oct 30 18:44:54.725597 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.725535 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ckbd9\"" Oct 30 18:44:54.725597 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.725548 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Oct 30 18:44:54.743392 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.743364 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.743501 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.743412 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b93de3d-8cc7-46e5-9d6b-501d33c802da-manager-config\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.743540 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.743508 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqjlt\" (UniqueName: \"kubernetes.io/projected/8b93de3d-8cc7-46e5-9d6b-501d33c802da-kube-api-access-rqjlt\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.743577 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.743555 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-metrics-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.844778 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.844753 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.844949 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.844791 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b93de3d-8cc7-46e5-9d6b-501d33c802da-manager-config\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.844949 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.844830 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqjlt\" (UniqueName: \"kubernetes.io/projected/8b93de3d-8cc7-46e5-9d6b-501d33c802da-kube-api-access-rqjlt\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.844949 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.844859 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-metrics-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.845628 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.845547 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b93de3d-8cc7-46e5-9d6b-501d33c802da-manager-config\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.847404 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.847383 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-metrics-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.847518 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.847452 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b93de3d-8cc7-46e5-9d6b-501d33c802da-cert\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.900919 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.900870 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqjlt\" (UniqueName: \"kubernetes.io/projected/8b93de3d-8cc7-46e5-9d6b-501d33c802da-kube-api-access-rqjlt\") pod \"lws-controller-manager-d5d6bc6bf-kf84g\" (UID: \"8b93de3d-8cc7-46e5-9d6b-501d33c802da\") " pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:54.935778 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:54.935755 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:55.296234 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:55.296195 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g"] Oct 30 18:44:55.298413 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:44:55.298384 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b93de3d_8cc7_46e5_9d6b_501d33c802da.slice/crio-7d2fb57ffe20edd0f2228b3d4229047df6c98f985cff383312e7f1265aa88a28 WatchSource:0}: Error finding container 7d2fb57ffe20edd0f2228b3d4229047df6c98f985cff383312e7f1265aa88a28: Status 404 returned error can't find the container with id 7d2fb57ffe20edd0f2228b3d4229047df6c98f985cff383312e7f1265aa88a28 Oct 30 18:44:55.708501 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:55.708451 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" event={"ID":"8b93de3d-8cc7-46e5-9d6b-501d33c802da","Type":"ContainerStarted","Data":"7d2fb57ffe20edd0f2228b3d4229047df6c98f985cff383312e7f1265aa88a28"} Oct 30 18:44:58.729451 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:58.729413 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" event={"ID":"8b93de3d-8cc7-46e5-9d6b-501d33c802da","Type":"ContainerStarted","Data":"c9068024fcb55641c68e0bac083cd72a34bc149f10f426b0de406e5286186829"} Oct 30 18:44:58.729872 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:58.729522 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:44:58.839634 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:44:58.839578 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" podStartSLOduration=2.02080687 podStartE2EDuration="4.8395628s" totalImagesPullingTime="2.81875593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:58.838858884 +0000 UTC m=+422.910682973" watchObservedRunningTime="2026-10-30 18:44:58.8395628 +0000 UTC m=+422.911386875" Oct 30 18:44:59.280845 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:44:59.280804 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:45:09.736136 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:09.736108 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d5d6bc6bf-kf84g" Oct 30 18:45:28.895892 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:28.895862 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-bm9bl"] Oct 30 18:45:28.903492 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:28.903473 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:28.916676 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:28.916649 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6s8t\" (UniqueName: \"kubernetes.io/projected/eed635c9-5674-4c53-93a0-9d0fd88eabfd-kube-api-access-c6s8t\") pod \"authorino-operator-66498fdf6c-bm9bl\" (UID: \"eed635c9-5674-4c53-93a0-9d0fd88eabfd\") " pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:28.957047 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:28.957022 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 18:45:28.958737 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:28.958721 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 18:45:29.004607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.004582 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8qxlm\"" Oct 30 18:45:29.006931 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.006912 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-bm9bl"] Oct 30 18:45:29.017520 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.017500 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6s8t\" (UniqueName: \"kubernetes.io/projected/eed635c9-5674-4c53-93a0-9d0fd88eabfd-kube-api-access-c6s8t\") pod \"authorino-operator-66498fdf6c-bm9bl\" (UID: \"eed635c9-5674-4c53-93a0-9d0fd88eabfd\") " pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:29.089970 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.089941 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6s8t\" (UniqueName: \"kubernetes.io/projected/eed635c9-5674-4c53-93a0-9d0fd88eabfd-kube-api-access-c6s8t\") pod \"authorino-operator-66498fdf6c-bm9bl\" (UID: \"eed635c9-5674-4c53-93a0-9d0fd88eabfd\") " pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:29.215935 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.215908 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:29.283415 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:45:29.283382 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:45:29.586984 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.586905 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-bm9bl"] Oct 30 18:45:29.590700 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:45:29.590674 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed635c9_5674_4c53_93a0_9d0fd88eabfd.slice/crio-f880294d41739499c95f01f4db539f9692f85d38cf8511e0d2398b080bb1a855 WatchSource:0}: Error finding container f880294d41739499c95f01f4db539f9692f85d38cf8511e0d2398b080bb1a855: Status 404 returned error can't find the container with id f880294d41739499c95f01f4db539f9692f85d38cf8511e0d2398b080bb1a855 Oct 30 18:45:29.855213 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:29.855150 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" event={"ID":"eed635c9-5674-4c53-93a0-9d0fd88eabfd","Type":"ContainerStarted","Data":"f880294d41739499c95f01f4db539f9692f85d38cf8511e0d2398b080bb1a855"} Oct 30 18:45:33.878123 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:33.878022 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" event={"ID":"eed635c9-5674-4c53-93a0-9d0fd88eabfd","Type":"ContainerStarted","Data":"2393e6fb1eb49be86e88e3c3971e340abbe1c3047038b86678ad7d7759ab7874"} Oct 30 18:45:33.878514 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:33.878183 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:33.991787 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:33.991736 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" podStartSLOduration=2.120264066 podStartE2EDuration="5.991724865s" totalImagesPullingTime="3.871460799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:33.991218126 +0000 UTC m=+458.063042204" watchObservedRunningTime="2026-10-30 18:45:33.991724865 +0000 UTC m=+458.063548941" Oct 30 18:45:44.885990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:45:44.885949 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-66498fdf6c-bm9bl" Oct 30 18:45:59.280107 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:45:59.280052 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:46:29.281758 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:46:29.281716 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:46:36.106910 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.106877 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:36.111607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.111584 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:36.162461 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.162435 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:36.208829 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.208808 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-69q5d\"" Oct 30 18:46:36.217354 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.217329 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frhl2\" (UniqueName: \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\") pod \"authorino-57f7f8659f-j964l\" (UID: \"a1b8932d-9ecc-42d1-ace7-cbad8430efa9\") " pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:36.318256 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.318225 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frhl2\" (UniqueName: \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\") pod \"authorino-57f7f8659f-j964l\" (UID: \"a1b8932d-9ecc-42d1-ace7-cbad8430efa9\") " pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:36.374218 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.374086 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frhl2\" (UniqueName: \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\") pod \"authorino-57f7f8659f-j964l\" (UID: \"a1b8932d-9ecc-42d1-ace7-cbad8430efa9\") " pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:36.423021 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.423002 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:36.784339 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:36.784311 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:36.787335 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:46:36.787306 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1b8932d_9ecc_42d1_ace7_cbad8430efa9.slice/crio-b148122c07ed457d14a9b50947161a4735278e031eae05767055c87fafe8505c WatchSource:0}: Error finding container b148122c07ed457d14a9b50947161a4735278e031eae05767055c87fafe8505c: Status 404 returned error can't find the container with id b148122c07ed457d14a9b50947161a4735278e031eae05767055c87fafe8505c Oct 30 18:46:37.146557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:37.146480 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-j964l" event={"ID":"a1b8932d-9ecc-42d1-ace7-cbad8430efa9","Type":"ContainerStarted","Data":"b148122c07ed457d14a9b50947161a4735278e031eae05767055c87fafe8505c"} Oct 30 18:46:38.865281 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:38.865254 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-grd5p"] Oct 30 18:46:38.868695 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:38.868677 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:38.922456 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:38.922417 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-grd5p"] Oct 30 18:46:38.968457 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:38.968432 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 30 18:46:39.018174 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.018147 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2s4f7\"" Oct 30 18:46:39.043693 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.043662 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08866c11-f350-4e1b-9548-629d7e67f0a0-config-file\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.043805 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.043713 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnspw\" (UniqueName: \"kubernetes.io/projected/08866c11-f350-4e1b-9548-629d7e67f0a0-kube-api-access-cnspw\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.144558 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.144492 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnspw\" (UniqueName: \"kubernetes.io/projected/08866c11-f350-4e1b-9548-629d7e67f0a0-kube-api-access-cnspw\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.144686 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.144559 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08866c11-f350-4e1b-9548-629d7e67f0a0-config-file\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.145194 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.145176 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08866c11-f350-4e1b-9548-629d7e67f0a0-config-file\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.169502 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.169481 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-grd5p"] Oct 30 18:46:39.209633 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.209604 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnspw\" (UniqueName: \"kubernetes.io/projected/08866c11-f350-4e1b-9548-629d7e67f0a0-kube-api-access-cnspw\") pod \"limitador-limitador-7c9b87d574-grd5p\" (UID: \"08866c11-f350-4e1b-9548-629d7e67f0a0\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.490198 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.490164 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:39.860131 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:39.860109 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-grd5p"] Oct 30 18:46:39.862029 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:46:39.862001 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08866c11_f350_4e1b_9548_629d7e67f0a0.slice/crio-6fa6787cfc3a38ac1a748aaea77d09a1e12564d633dd324e235d74c7b0251cd3 WatchSource:0}: Error finding container 6fa6787cfc3a38ac1a748aaea77d09a1e12564d633dd324e235d74c7b0251cd3: Status 404 returned error can't find the container with id 6fa6787cfc3a38ac1a748aaea77d09a1e12564d633dd324e235d74c7b0251cd3 Oct 30 18:46:40.160173 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:40.160082 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-j964l" event={"ID":"a1b8932d-9ecc-42d1-ace7-cbad8430efa9","Type":"ContainerStarted","Data":"3ba4a07f638c9b9f423242b4676d178f6c5da0bfbf97bd41b810ab41b7b65da1"} Oct 30 18:46:40.161319 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:40.161299 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" event={"ID":"08866c11-f350-4e1b-9548-629d7e67f0a0","Type":"ContainerStarted","Data":"6fa6787cfc3a38ac1a748aaea77d09a1e12564d633dd324e235d74c7b0251cd3"} Oct 30 18:46:40.268735 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:40.268689 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-j964l" podStartSLOduration=1.295135941 podStartE2EDuration="4.26867662s" totalImagesPullingTime="2.973540679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:40.268234083 +0000 UTC m=+524.340058160" watchObservedRunningTime="2026-10-30 18:46:40.26867662 +0000 UTC m=+524.340500686" Oct 30 18:46:42.171583 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:42.171545 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" event={"ID":"08866c11-f350-4e1b-9548-629d7e67f0a0","Type":"ContainerStarted","Data":"3df01f50f0133814bcaf27e0e807b7a8821cf659a6b083ddee2f3585138781e1"} Oct 30 18:46:42.171972 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:42.171648 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:42.282249 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:42.282199 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" podStartSLOduration=2.474420943 podStartE2EDuration="4.282185086s" totalImagesPullingTime="1.807764143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:42.28140929 +0000 UTC m=+526.353233367" watchObservedRunningTime="2026-10-30 18:46:42.282185086 +0000 UTC m=+526.354009159" Oct 30 18:46:44.993655 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:44.993622 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:44.994132 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:44.993823 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-j964l" podUID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" containerName="authorino" containerID="cri-o://3ba4a07f638c9b9f423242b4676d178f6c5da0bfbf97bd41b810ab41b7b65da1" gracePeriod=30 Oct 30 18:46:45.185611 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.185583 5869 generic.go:356] "Generic (PLEG): container finished" podID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" containerID="3ba4a07f638c9b9f423242b4676d178f6c5da0bfbf97bd41b810ab41b7b65da1" exitCode=0 Oct 30 18:46:45.185734 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.185636 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-j964l" event={"ID":"a1b8932d-9ecc-42d1-ace7-cbad8430efa9","Type":"ContainerDied","Data":"3ba4a07f638c9b9f423242b4676d178f6c5da0bfbf97bd41b810ab41b7b65da1"} Oct 30 18:46:45.237358 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.237334 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:45.293855 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.293799 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\" (UniqueName: \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\") pod \"a1b8932d-9ecc-42d1-ace7-cbad8430efa9\" (UID: \"a1b8932d-9ecc-42d1-ace7-cbad8430efa9\") " Oct 30 18:46:45.295922 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.295897 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2" pod "a1b8932d-9ecc-42d1-ace7-cbad8430efa9" (UID: "a1b8932d-9ecc-42d1-ace7-cbad8430efa9"). InnerVolumeSpecName "kube-api-access-frhl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:45.395170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:45.395144 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frhl2\" (UniqueName: \"kubernetes.io/projected/a1b8932d-9ecc-42d1-ace7-cbad8430efa9-kube-api-access-frhl2\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:46:46.191270 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:46.191238 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-j964l" Oct 30 18:46:46.191666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:46.191264 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-j964l" event={"ID":"a1b8932d-9ecc-42d1-ace7-cbad8430efa9","Type":"ContainerDied","Data":"b148122c07ed457d14a9b50947161a4735278e031eae05767055c87fafe8505c"} Oct 30 18:46:46.191666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:46.191306 5869 scope.go:122] "RemoveContainer" containerID="3ba4a07f638c9b9f423242b4676d178f6c5da0bfbf97bd41b810ab41b7b65da1" Oct 30 18:46:46.348820 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:46.348761 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:46.351874 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:46.351850 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-j964l"] Oct 30 18:46:48.041508 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:48.041471 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" path="/var/lib/kubelet/pods/a1b8932d-9ecc-42d1-ace7-cbad8430efa9/volumes" Oct 30 18:46:53.176572 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:46:53.176538 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-grd5p" Oct 30 18:46:59.280579 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:46:59.280541 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:47:06.089438 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.089364 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557c8f8584-z67lc"] Oct 30 18:47:06.093214 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.089741 5869 cpu_manager.go:414] "containerMap: removing container" podUID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" containerName="authorino" Oct 30 18:47:06.093214 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.089753 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" containerName="authorino" Oct 30 18:47:06.093214 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.089823 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1b8932d-9ecc-42d1-ace7-cbad8430efa9" containerName="authorino" Oct 30 18:47:06.094463 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.094447 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.148124 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.148080 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-z67lc"] Oct 30 18:47:06.155864 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.155828 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtvbt\" (UniqueName: \"kubernetes.io/projected/d28ef0aa-fac8-418d-a1bd-cb9967713197-kube-api-access-vtvbt\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.155972 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.155931 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d28ef0aa-fac8-418d-a1bd-cb9967713197-tls-cert\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.194190 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.194165 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Oct 30 18:47:06.194190 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.194165 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-69q5d\"" Oct 30 18:47:06.256843 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.256816 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d28ef0aa-fac8-418d-a1bd-cb9967713197-tls-cert\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.256940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.256889 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtvbt\" (UniqueName: \"kubernetes.io/projected/d28ef0aa-fac8-418d-a1bd-cb9967713197-kube-api-access-vtvbt\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.259453 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.259435 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d28ef0aa-fac8-418d-a1bd-cb9967713197-tls-cert\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.313057 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.313030 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtvbt\" (UniqueName: \"kubernetes.io/projected/d28ef0aa-fac8-418d-a1bd-cb9967713197-kube-api-access-vtvbt\") pod \"authorino-557c8f8584-z67lc\" (UID: \"d28ef0aa-fac8-418d-a1bd-cb9967713197\") " pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.406993 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.406939 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-z67lc" Oct 30 18:47:06.771960 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:06.771923 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-z67lc"] Oct 30 18:47:06.774280 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:47:06.774250 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd28ef0aa_fac8_418d_a1bd_cb9967713197.slice/crio-7a313ce1969a1e4c70ccf15fcb59332e36042940a400d1c281367addb86a037a WatchSource:0}: Error finding container 7a313ce1969a1e4c70ccf15fcb59332e36042940a400d1c281367addb86a037a: Status 404 returned error can't find the container with id 7a313ce1969a1e4c70ccf15fcb59332e36042940a400d1c281367addb86a037a Oct 30 18:47:07.284622 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:07.284584 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-z67lc" event={"ID":"d28ef0aa-fac8-418d-a1bd-cb9967713197","Type":"ContainerStarted","Data":"7a313ce1969a1e4c70ccf15fcb59332e36042940a400d1c281367addb86a037a"} Oct 30 18:47:09.294845 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:09.294811 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-z67lc" event={"ID":"d28ef0aa-fac8-418d-a1bd-cb9967713197","Type":"ContainerStarted","Data":"a6b6f0fdd50ab6982ba471e2ddc0e6f89e3661cc88f1102a614b1033586b72b5"} Oct 30 18:47:09.405937 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:47:09.405880 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-557c8f8584-z67lc" podStartSLOduration=1.6120824809999998 podStartE2EDuration="3.405867233s" totalImagesPullingTime="1.793784752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:09.405117323 +0000 UTC m=+553.476941400" watchObservedRunningTime="2026-10-30 18:47:09.405867233 +0000 UTC m=+553.477691342" Oct 30 18:47:29.280888 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:47:29.280847 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:47:59.284337 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:47:59.284297 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:48:29.281805 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:48:29.281767 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:48:59.283612 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:48:59.283573 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:49:29.282306 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:49:29.282263 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:49:59.281019 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:49:59.280974 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:50:29.281834 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:50:29.281781 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:50:59.282444 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:50:59.282403 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:51:29.281164 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:51:29.281127 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:51:30.122792 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.122754 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:51:30.126571 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.126547 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.170318 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.170289 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.170450 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.170334 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6n7m\" (UniqueName: \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.180821 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.180785 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:51:30.227526 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.227504 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-hqvzl\"" Oct 30 18:51:30.227526 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.227522 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\"" Oct 30 18:51:30.227666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.227506 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-epp-sa-dockercfg-swrrp\"" Oct 30 18:51:30.227666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.227526 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\"" Oct 30 18:51:30.227666 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.227499 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\"" Oct 30 18:51:30.271740 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.271716 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.271834 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.271754 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6n7m\" (UniqueName: \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.274145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.274114 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.328399 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.328372 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6n7m\" (UniqueName: \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.460945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.460928 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:30.828168 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.828143 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:51:30.830844 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:51:30.830814 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56425975_af46_496e_8820_6d9493d0d8f9.slice/crio-5109ffb50d0c6031b5005fd8e631f78e8ff5bc2b210a487d1b17a5412f49d18c WatchSource:0}: Error finding container 5109ffb50d0c6031b5005fd8e631f78e8ff5bc2b210a487d1b17a5412f49d18c: Status 404 returned error can't find the container with id 5109ffb50d0c6031b5005fd8e631f78e8ff5bc2b210a487d1b17a5412f49d18c Oct 30 18:51:30.832911 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:30.832894 5869 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:51:31.454715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:31.454675 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" event={"ID":"56425975-af46-496e-8820-6d9493d0d8f9","Type":"ContainerStarted","Data":"5109ffb50d0c6031b5005fd8e631f78e8ff5bc2b210a487d1b17a5412f49d18c"} Oct 30 18:51:33.464507 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:33.464465 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" event={"ID":"56425975-af46-496e-8820-6d9493d0d8f9","Type":"ContainerStarted","Data":"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f"} Oct 30 18:51:33.464875 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:33.464551 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:51:33.576344 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:33.576297 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podStartSLOduration=1.351209244 podStartE2EDuration="3.576284447s" totalImagesPullingTime="2.225075203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:33.57545585 +0000 UTC m=+817.647279927" watchObservedRunningTime="2026-10-30 18:51:33.576284447 +0000 UTC m=+817.648108522" Oct 30 18:51:47.971289 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:47.971247 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn"] Oct 30 18:51:47.975250 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:47.975231 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.077998 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.077932 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn"] Oct 30 18:51:48.093062 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.093042 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-dstgc\"" Oct 30 18:51:48.094281 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.094259 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:51:48.094417 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.094363 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 18:51:48.113248 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113227 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113347 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113256 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113347 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113280 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkmbl\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-kube-api-access-xkmbl\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113347 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113328 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113474 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113370 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113474 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113400 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113474 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113418 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113592 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113505 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.113592 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.113525 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.123193 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.123169 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:51:48.213822 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213794 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.213940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213832 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.213940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213862 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.213940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213893 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.213940 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213929 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkmbl\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-kube-api-access-xkmbl\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214201 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.213978 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214201 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214016 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214313 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214086 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214313 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214266 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214424 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214371 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214478 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214442 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214617 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214556 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214617 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214607 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.214890 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.214879 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.216385 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.216364 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.216727 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.216709 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.270655 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.270603 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.271020 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.270997 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkmbl\" (UniqueName: \"kubernetes.io/projected/bfb6eabc-9dde-46b5-bcae-cf775eaeed51-kube-api-access-xkmbl\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-2dzfn\" (UID: \"bfb6eabc-9dde-46b5-bcae-cf775eaeed51\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.288167 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.288151 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:48.718785 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:48.718755 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn"] Oct 30 18:51:48.721498 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:51:48.721473 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb6eabc_9dde_46b5_bcae_cf775eaeed51.slice/crio-c86e2d963be8f72af3d51f02df171b2dd69c93965fc8e37d9ffcc82d8dbdf280 WatchSource:0}: Error finding container c86e2d963be8f72af3d51f02df171b2dd69c93965fc8e37d9ffcc82d8dbdf280: Status 404 returned error can't find the container with id c86e2d963be8f72af3d51f02df171b2dd69c93965fc8e37d9ffcc82d8dbdf280 Oct 30 18:51:49.537912 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:49.537877 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" event={"ID":"bfb6eabc-9dde-46b5-bcae-cf775eaeed51","Type":"ContainerStarted","Data":"c86e2d963be8f72af3d51f02df171b2dd69c93965fc8e37d9ffcc82d8dbdf280"} Oct 30 18:51:53.078785 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:53.078758 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:51:53.079071 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:53.078822 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:51:53.079071 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:53.078850 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:51:53.559142 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:53.559077 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" event={"ID":"bfb6eabc-9dde-46b5-bcae-cf775eaeed51","Type":"ContainerStarted","Data":"4d61212a7874e59920e492b51f0a6ca49e032f63038cecc1ac6067e2f6ab0abf"} Oct 30 18:51:53.721031 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:53.720970 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podStartSLOduration=2.366621056 podStartE2EDuration="6.720951933s" totalImagesPullingTime="4.354330877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:53.720492195 +0000 UTC m=+837.792316270" watchObservedRunningTime="2026-10-30 18:51:53.720951933 +0000 UTC m=+837.792776010" Oct 30 18:51:54.289469 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:54.289434 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:54.293135 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:54.292746 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Oct 30 18:51:54.293135 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:54.292864 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podUID="bfb6eabc-9dde-46b5-bcae-cf775eaeed51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Oct 30 18:51:55.293469 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:55.293435 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Oct 30 18:51:55.293883 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:55.293487 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podUID="bfb6eabc-9dde-46b5-bcae-cf775eaeed51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Oct 30 18:51:56.292431 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:56.292399 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Oct 30 18:51:56.292715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:56.292687 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podUID="bfb6eabc-9dde-46b5-bcae-cf775eaeed51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Oct 30 18:51:57.293432 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:57.293398 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Oct 30 18:51:57.293892 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:57.293457 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podUID="bfb6eabc-9dde-46b5-bcae-cf775eaeed51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Oct 30 18:51:58.292118 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:58.291407 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Oct 30 18:51:58.292118 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:58.291466 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" podUID="bfb6eabc-9dde-46b5-bcae-cf775eaeed51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Oct 30 18:51:59.279482 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:51:59.279445 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:51:59.292380 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:59.292358 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:59.292670 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:59.292643 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:51:59.293258 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:51:59.293237 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-2dzfn" Oct 30 18:52:04.471257 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:04.471226 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:52:11.411532 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:11.411500 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:52:14.471632 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:14.471595 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:52:24.471702 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:24.471657 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:52:26.431808 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:26.431763 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" containerID="cri-o://34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f" gracePeriod=45 Oct 30 18:52:29.283045 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:52:29.282996 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:52:32.916164 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.916130 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 18:52:32.925642 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.925620 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.992820 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.992797 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.992945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.992828 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x9cm\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.992945 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.992864 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993026 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.992968 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993026 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.993005 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993026 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.993022 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993195 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.993118 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993195 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.993145 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:32.993195 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:32.993169 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.008058 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.008035 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 18:52:33.094269 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094246 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094399 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094272 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094399 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094292 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094399 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094311 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094444 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094498 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094557 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094529 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x9cm\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094606 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094657 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094826 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094754 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094826 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094811 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.094928 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094876 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.095001 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.094981 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.095375 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.095354 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.096611 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.096586 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.096920 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.096900 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.150030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.150004 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.150202 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.150186 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x9cm\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-tdnkf\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.238837 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.238815 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:33.652584 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.652545 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 18:52:33.654892 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:52:33.654861 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d9cb1f_4b80_4bc0_97e4_7e4464117328.slice/crio-217a4315087d7680da61a0d1c1ea1c45e957eb3be727b03f3cdf1c0209ae35cc WatchSource:0}: Error finding container 217a4315087d7680da61a0d1c1ea1c45e957eb3be727b03f3cdf1c0209ae35cc: Status 404 returned error can't find the container with id 217a4315087d7680da61a0d1c1ea1c45e957eb3be727b03f3cdf1c0209ae35cc Oct 30 18:52:33.657067 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.657036 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:52:33.657258 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.657131 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:52:33.657258 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.657178 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:52:33.759189 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.759153 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerStarted","Data":"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643"} Oct 30 18:52:33.759339 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.759212 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerStarted","Data":"217a4315087d7680da61a0d1c1ea1c45e957eb3be727b03f3cdf1c0209ae35cc"} Oct 30 18:52:33.880912 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:33.880864 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podStartSLOduration=1.880849472 podStartE2EDuration="1.880849472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:33.87970528 +0000 UTC m=+877.951529387" watchObservedRunningTime="2026-10-30 18:52:33.880849472 +0000 UTC m=+877.952673548" Oct 30 18:52:34.239891 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.239857 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:34.244612 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.244587 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:34.471217 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.471174 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:52:34.471383 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.471266 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:52:34.763427 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.763399 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:34.764364 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:34.764346 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:52:44.471382 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:44.471348 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:52:54.471300 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:54.471261 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:52:56.684829 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.684800 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:52:56.790375 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.790295 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\" (UniqueName: \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\") pod \"56425975-af46-496e-8820-6d9493d0d8f9\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " Oct 30 18:52:56.790375 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.790347 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\" (UniqueName: \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\") pod \"56425975-af46-496e-8820-6d9493d0d8f9\" (UID: \"56425975-af46-496e-8820-6d9493d0d8f9\") " Oct 30 18:52:56.792694 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.792652 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs" pod "56425975-af46-496e-8820-6d9493d0d8f9" (UID: "56425975-af46-496e-8820-6d9493d0d8f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:52:56.792811 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.792735 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m" pod "56425975-af46-496e-8820-6d9493d0d8f9" (UID: "56425975-af46-496e-8820-6d9493d0d8f9"). InnerVolumeSpecName "kube-api-access-r6n7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:52:56.860727 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.860698 5869 generic.go:356] "Generic (PLEG): container finished" podID="56425975-af46-496e-8820-6d9493d0d8f9" containerID="34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f" exitCode=0 Oct 30 18:52:56.860870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.860734 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" event={"ID":"56425975-af46-496e-8820-6d9493d0d8f9","Type":"ContainerDied","Data":"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f"} Oct 30 18:52:56.860870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.860744 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" Oct 30 18:52:56.860870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.860770 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27" event={"ID":"56425975-af46-496e-8820-6d9493d0d8f9","Type":"ContainerDied","Data":"5109ffb50d0c6031b5005fd8e631f78e8ff5bc2b210a487d1b17a5412f49d18c"} Oct 30 18:52:56.860870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.860791 5869 scope.go:122] "RemoveContainer" containerID="34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f" Oct 30 18:52:56.875872 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.875852 5869 scope.go:122] "RemoveContainer" containerID="34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f" Oct 30 18:52:56.876164 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:52:56.876144 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f\": container with ID starting with 34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f not found: ID does not exist" containerID="34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f" Oct 30 18:52:56.876224 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.876173 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f"} err="failed to get container status \"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f\": rpc error: code = NotFound desc = could not find container \"34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f\": container with ID starting with 34f603965ab568fd9e33afff7d8f302ae0e31300dd075c4a13d47bad5f51c17f not found: ID does not exist" Oct 30 18:52:56.891354 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.891332 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6n7m\" (UniqueName: \"kubernetes.io/projected/56425975-af46-496e-8820-6d9493d0d8f9-kube-api-access-r6n7m\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:52:56.891354 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:56.891355 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56425975-af46-496e-8820-6d9493d0d8f9-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:52:57.019050 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:57.019021 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:52:57.023685 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:57.023664 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859b4pl27"] Oct 30 18:52:58.040773 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:52:58.040741 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56425975-af46-496e-8820-6d9493d0d8f9" path="/var/lib/kubelet/pods/56425975-af46-496e-8820-6d9493d0d8f9/volumes" Oct 30 18:52:59.279679 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:52:59.279645 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:53:06.840881 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.840806 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:53:06.841285 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.841265 5869 cpu_manager.go:414] "containerMap: removing container" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" Oct 30 18:53:06.841285 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.841280 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" Oct 30 18:53:06.841356 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.841341 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="56425975-af46-496e-8820-6d9493d0d8f9" containerName="main" Oct 30 18:53:06.846143 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.846115 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:06.903990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.903964 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:53:06.911814 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.911796 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\"" Oct 30 18:53:06.943633 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.943610 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\"" Oct 30 18:53:06.943750 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.943723 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-mx5ss\"" Oct 30 18:53:06.943822 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.943773 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-9lgj9\"" Oct 30 18:53:06.944326 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.944314 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\"" Oct 30 18:53:06.967588 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.967566 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:06.967677 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:06.967625 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vshr4\" (UniqueName: \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.068944 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.068905 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vshr4\" (UniqueName: \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.069089 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.068981 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.071501 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.071480 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.127361 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.127307 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vshr4\" (UniqueName: \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.158153 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.158122 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:07.522227 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.522203 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:53:07.524921 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:53:07.524882 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cac55da_0cce_49c5_bd9e_e85a57638fe0.slice/crio-27b17526d6b0c152655ef1ce55ddd93041e876211edb1f8917ca051bf83b4ddb WatchSource:0}: Error finding container 27b17526d6b0c152655ef1ce55ddd93041e876211edb1f8917ca051bf83b4ddb: Status 404 returned error can't find the container with id 27b17526d6b0c152655ef1ce55ddd93041e876211edb1f8917ca051bf83b4ddb Oct 30 18:53:07.911507 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.911432 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" event={"ID":"9cac55da-0cce-49c5-bd9e-e85a57638fe0","Type":"ContainerStarted","Data":"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9"} Oct 30 18:53:07.911507 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.911481 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" event={"ID":"9cac55da-0cce-49c5-bd9e-e85a57638fe0","Type":"ContainerStarted","Data":"27b17526d6b0c152655ef1ce55ddd93041e876211edb1f8917ca051bf83b4ddb"} Oct 30 18:53:07.911898 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:07.911523 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:08.022550 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:08.022495 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podStartSLOduration=2.022478863 podStartE2EDuration="2.022478863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:08.021534652 +0000 UTC m=+912.093358730" watchObservedRunningTime="2026-10-30 18:53:08.022478863 +0000 UTC m=+912.094302943" Oct 30 18:53:18.962597 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.962567 5869 generic.go:356] "Generic (PLEG): container finished" podID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerID="e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643" exitCode=0 Oct 30 18:53:18.962990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.962628 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerDied","Data":"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643"} Oct 30 18:53:18.962990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.962973 5869 scope.go:122] "RemoveContainer" containerID="e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643" Oct 30 18:53:18.964127 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.964081 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:18.964208 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.964173 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:18.964245 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:18.964211 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:19.763740 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:19.763698 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:53:19.995342 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:19.995305 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerStarted","Data":"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a"} Oct 30 18:53:20.239076 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:20.239038 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:53:20.239985 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:20.239959 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Oct 30 18:53:20.240112 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:20.240005 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Oct 30 18:53:21.239412 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:21.239374 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Oct 30 18:53:21.239801 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:21.239430 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Oct 30 18:53:22.239775 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:22.239736 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Oct 30 18:53:22.240192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:22.239795 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Oct 30 18:53:23.239952 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:23.239917 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Oct 30 18:53:23.240381 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:23.239969 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Oct 30 18:53:24.239416 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:24.239382 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Oct 30 18:53:24.239589 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:24.239442 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Oct 30 18:53:25.243282 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:25.243253 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:53:25.243675 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:25.243520 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:53:25.244283 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:25.244265 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 18:53:29.279052 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:53:29.279013 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:53:33.171569 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.171540 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 18:53:33.175038 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.175022 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214333 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214302 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214470 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214336 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbkm9\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214470 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214356 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214470 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214437 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214630 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214544 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214684 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214635 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214684 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214664 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214776 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214695 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.214810 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.214786 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315690 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315715 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315720 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315738 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315765 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315782 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315801 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbkm9\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315830 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315868 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.315929 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.315929 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.316296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.316125 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.316296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.316169 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.316618 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.316600 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.316757 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.316737 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.317013 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.316990 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.318612 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.318594 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.318673 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.318594 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.363909 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.363877 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 18:53:33.374988 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.374958 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.375620 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.375598 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbkm9\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-lb64l\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.488011 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.487987 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:33.947294 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.947270 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 18:53:33.949629 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:53:33.949600 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21472f59_810b_4571_8e3b_eb160c4ff080.slice/crio-ad2d54fe442196a2d988fdc8218c278b2b5388bf430482075b305935583c7fa2 WatchSource:0}: Error finding container ad2d54fe442196a2d988fdc8218c278b2b5388bf430482075b305935583c7fa2: Status 404 returned error can't find the container with id ad2d54fe442196a2d988fdc8218c278b2b5388bf430482075b305935583c7fa2 Oct 30 18:53:33.951628 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.951597 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:33.951707 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.951663 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:33.951707 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:33.951691 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 18:53:34.058176 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:34.058142 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" event={"ID":"21472f59-810b-4571-8e3b-eb160c4ff080","Type":"ContainerStarted","Data":"2b59ab624ba9f0e5de6e624bb8ad2853656b9f7a53423ec522c8c464dcb2e645"} Oct 30 18:53:34.058324 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:34.058203 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" event={"ID":"21472f59-810b-4571-8e3b-eb160c4ff080","Type":"ContainerStarted","Data":"ad2d54fe442196a2d988fdc8218c278b2b5388bf430482075b305935583c7fa2"} Oct 30 18:53:34.172415 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:34.172365 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podStartSLOduration=1.172350932 podStartE2EDuration="1.172350932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:34.171444356 +0000 UTC m=+938.243268435" watchObservedRunningTime="2026-10-30 18:53:34.172350932 +0000 UTC m=+938.244175002" Oct 30 18:53:34.488256 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:34.488199 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:35.493242 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:35.493202 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-lb64l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Oct 30 18:53:35.493779 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:35.493260 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Oct 30 18:53:36.493494 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:36.493460 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-lb64l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Oct 30 18:53:36.494063 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:36.494029 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Oct 30 18:53:37.493233 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:37.493192 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-lb64l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Oct 30 18:53:37.493504 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:37.493260 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Oct 30 18:53:38.488543 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:38.488511 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-lb64l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Oct 30 18:53:38.489170 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:38.489138 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Oct 30 18:53:38.918240 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:38.918166 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:53:39.493236 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:39.493195 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-lb64l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Oct 30 18:53:39.493722 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:39.493278 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Oct 30 18:53:40.492061 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:40.492030 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:40.492346 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:40.492327 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:40.492920 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:40.492905 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 18:53:45.504590 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:45.497268 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:53:48.918307 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:48.918271 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:53:58.918067 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:53:58.918032 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:53:59.279781 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:53:59.279743 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:54:00.570983 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:00.570944 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" containerID="cri-o://5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9" gracePeriod=45 Oct 30 18:54:08.918262 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:08.918227 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:54:08.918657 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:08.918327 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:54:18.918289 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:18.918255 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:54:28.917502 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:28.917465 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:54:29.279672 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:54:29.279635 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:54:30.818491 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:30.818468 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:54:30.931066 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:30.930982 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\") pod \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " Oct 30 18:54:30.931318 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:30.931291 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\" (UniqueName: \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\") pod \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\" (UID: \"9cac55da-0cce-49c5-bd9e-e85a57638fe0\") " Oct 30 18:54:30.933459 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:30.933430 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs" pod "9cac55da-0cce-49c5-bd9e-e85a57638fe0" (UID: "9cac55da-0cce-49c5-bd9e-e85a57638fe0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:54:30.933778 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:30.933755 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4" pod "9cac55da-0cce-49c5-bd9e-e85a57638fe0" (UID: "9cac55da-0cce-49c5-bd9e-e85a57638fe0"). InnerVolumeSpecName "kube-api-access-vshr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:54:31.032644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.032621 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac55da-0cce-49c5-bd9e-e85a57638fe0-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:54:31.032644 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.032644 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vshr4\" (UniqueName: \"kubernetes.io/projected/9cac55da-0cce-49c5-bd9e-e85a57638fe0-kube-api-access-vshr4\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:54:31.327448 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.327418 5869 generic.go:356] "Generic (PLEG): container finished" podID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerID="5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9" exitCode=0 Oct 30 18:54:31.327582 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.327462 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" Oct 30 18:54:31.327582 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.327462 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" event={"ID":"9cac55da-0cce-49c5-bd9e-e85a57638fe0","Type":"ContainerDied","Data":"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9"} Oct 30 18:54:31.327656 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.327590 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv" event={"ID":"9cac55da-0cce-49c5-bd9e-e85a57638fe0","Type":"ContainerDied","Data":"27b17526d6b0c152655ef1ce55ddd93041e876211edb1f8917ca051bf83b4ddb"} Oct 30 18:54:31.327656 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.327623 5869 scope.go:122] "RemoveContainer" containerID="5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9" Oct 30 18:54:31.336374 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.336354 5869 scope.go:122] "RemoveContainer" containerID="5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9" Oct 30 18:54:31.336615 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:54:31.336597 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9\": container with ID starting with 5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9 not found: ID does not exist" containerID="5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9" Oct 30 18:54:31.336668 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.336624 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9"} err="failed to get container status \"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9\": rpc error: code = NotFound desc = could not find container \"5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9\": container with ID starting with 5a5c64c105bba8a9947d412cbf0de6d49446feee669a11cb7411eb82f90698d9 not found: ID does not exist" Oct 30 18:54:31.485581 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.485555 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:54:31.532870 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:31.532847 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schefqvpv"] Oct 30 18:54:32.041068 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:32.041040 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" path="/var/lib/kubelet/pods/9cac55da-0cce-49c5-bd9e-e85a57638fe0/volumes" Oct 30 18:54:38.693608 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.693574 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:54:38.694009 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.693950 5869 cpu_manager.go:414] "containerMap: removing container" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" Oct 30 18:54:38.694009 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.693960 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" Oct 30 18:54:38.694108 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.694016 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="9cac55da-0cce-49c5-bd9e-e85a57638fe0" containerName="main" Oct 30 18:54:38.698548 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.698528 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.750758 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.750730 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:54:38.796607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.796586 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-jcj45\"" Oct 30 18:54:38.796607 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.796597 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\"" Oct 30 18:54:38.796817 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.796800 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-4k49r\"" Oct 30 18:54:38.796857 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.796808 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\"" Oct 30 18:54:38.796857 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.796821 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\"" Oct 30 18:54:38.799145 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.799115 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.799271 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.799254 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbqb\" (UniqueName: \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.900051 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.900023 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbqb\" (UniqueName: \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.900231 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.900088 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.902761 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.902738 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:38.955981 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:38.955904 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbqb\" (UniqueName: \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:39.010858 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:39.010834 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:39.373760 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:39.373660 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:54:39.376113 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:54:39.376066 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00f70c2_b25a_4aee_9d8d_f8a7906e4430.slice/crio-2e204ea8b4f0075d5aab9d7053349a73c82c49e0c472699cc0694e1420bdf45c WatchSource:0}: Error finding container 2e204ea8b4f0075d5aab9d7053349a73c82c49e0c472699cc0694e1420bdf45c: Status 404 returned error can't find the container with id 2e204ea8b4f0075d5aab9d7053349a73c82c49e0c472699cc0694e1420bdf45c Oct 30 18:54:40.370349 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:40.370315 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" event={"ID":"c00f70c2-b25a-4aee-9d8d-f8a7906e4430","Type":"ContainerStarted","Data":"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019"} Oct 30 18:54:40.370739 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:40.370367 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" event={"ID":"c00f70c2-b25a-4aee-9d8d-f8a7906e4430","Type":"ContainerStarted","Data":"2e204ea8b4f0075d5aab9d7053349a73c82c49e0c472699cc0694e1420bdf45c"} Oct 30 18:54:40.370739 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:40.370406 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:54:40.482800 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:54:40.482750 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podStartSLOduration=2.482736695 podStartE2EDuration="2.482736695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:40.481922422 +0000 UTC m=+1004.553746499" watchObservedRunningTime="2026-10-30 18:54:40.482736695 +0000 UTC m=+1004.554560770" Oct 30 18:54:59.281634 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:54:59.281592 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:55:11.376233 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:11.376202 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:55:18.402219 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:18.402184 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:55:21.376280 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:21.376233 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:55:29.279452 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:55:29.279411 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:55:31.376623 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:31.376583 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:55:33.426859 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:33.426790 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" containerID="cri-o://75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019" gracePeriod=45 Oct 30 18:55:41.376995 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:41.376957 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:55:41.377627 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:41.377600 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:55:51.376629 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:55:51.376589 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:55:59.279626 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:55:59.279585 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:56:01.375668 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:01.375627 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:56:03.676579 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.676552 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:56:03.722427 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.722382 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\" (UniqueName: \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\") pod \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " Oct 30 18:56:03.722524 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.722432 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\") pod \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\" (UID: \"c00f70c2-b25a-4aee-9d8d-f8a7906e4430\") " Oct 30 18:56:03.724689 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.724650 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs" pod "c00f70c2-b25a-4aee-9d8d-f8a7906e4430" (UID: "c00f70c2-b25a-4aee-9d8d-f8a7906e4430"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:56:03.724786 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.724709 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb" pod "c00f70c2-b25a-4aee-9d8d-f8a7906e4430" (UID: "c00f70c2-b25a-4aee-9d8d-f8a7906e4430"). InnerVolumeSpecName "kube-api-access-frbqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:56:03.733107 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.733071 5869 generic.go:356] "Generic (PLEG): container finished" podID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerID="75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019" exitCode=0 Oct 30 18:56:03.733192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.733133 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" Oct 30 18:56:03.733192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.733134 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" event={"ID":"c00f70c2-b25a-4aee-9d8d-f8a7906e4430","Type":"ContainerDied","Data":"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019"} Oct 30 18:56:03.733192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.733187 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn" event={"ID":"c00f70c2-b25a-4aee-9d8d-f8a7906e4430","Type":"ContainerDied","Data":"2e204ea8b4f0075d5aab9d7053349a73c82c49e0c472699cc0694e1420bdf45c"} Oct 30 18:56:03.733318 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.733211 5869 scope.go:122] "RemoveContainer" containerID="75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019" Oct 30 18:56:03.741606 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.741588 5869 scope.go:122] "RemoveContainer" containerID="75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019" Oct 30 18:56:03.741870 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:56:03.741853 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019\": container with ID starting with 75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019 not found: ID does not exist" containerID="75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019" Oct 30 18:56:03.741915 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.741878 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019"} err="failed to get container status \"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019\": rpc error: code = NotFound desc = could not find container \"75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019\": container with ID starting with 75d7c65790c86482a4e1214464a6c9ddc114961f9f8ed71efb0ea2e2649a9019 not found: ID does not exist" Oct 30 18:56:03.823305 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.823278 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frbqb\" (UniqueName: \"kubernetes.io/projected/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-kube-api-access-frbqb\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:03.823305 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.823302 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00f70c2-b25a-4aee-9d8d-f8a7906e4430-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:03.895688 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.895658 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:56:03.901030 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:03.901009 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599b7m5jn"] Oct 30 18:56:04.040937 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:04.040878 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" path="/var/lib/kubelet/pods/c00f70c2-b25a-4aee-9d8d-f8a7906e4430/volumes" Oct 30 18:56:15.941652 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.941566 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:56:15.942161 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.942146 5869 cpu_manager.go:414] "containerMap: removing container" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" Oct 30 18:56:15.942252 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.942165 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" Oct 30 18:56:15.942316 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.942287 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="c00f70c2-b25a-4aee-9d8d-f8a7906e4430" containerName="main" Oct 30 18:56:15.945290 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.945268 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:15.999918 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:15.999890 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:56:16.017232 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.017204 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.017364 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.017345 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bpw\" (UniqueName: \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.045735 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.045714 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-2nbtr\"" Oct 30 18:56:16.045878 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.045801 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-x4x8q\"" Oct 30 18:56:16.046519 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.046496 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\"" Oct 30 18:56:16.046519 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.046505 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\"" Oct 30 18:56:16.046638 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.046551 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\"" Oct 30 18:56:16.117825 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.117794 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.117974 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.117904 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bpw\" (UniqueName: \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.120504 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.120484 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.173800 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.173778 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bpw\" (UniqueName: \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.258239 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.258185 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.621507 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.621486 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:56:16.624276 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:56:16.624251 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b03a64_8755_404f_b3fb_ca2ed66b9e7d.slice/crio-7b006b3cff3746b95db9b2973e46158e41114ace13ddfdd78533774a39d89362 WatchSource:0}: Error finding container 7b006b3cff3746b95db9b2973e46158e41114ace13ddfdd78533774a39d89362: Status 404 returned error can't find the container with id 7b006b3cff3746b95db9b2973e46158e41114ace13ddfdd78533774a39d89362 Oct 30 18:56:16.789799 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.789767 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" event={"ID":"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d","Type":"ContainerStarted","Data":"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb"} Oct 30 18:56:16.789990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.789820 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" event={"ID":"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d","Type":"ContainerStarted","Data":"7b006b3cff3746b95db9b2973e46158e41114ace13ddfdd78533774a39d89362"} Oct 30 18:56:16.789990 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.789884 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:16.900685 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:16.900587 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podStartSLOduration=1.900573943 podStartE2EDuration="1.900573943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:16.899340063 +0000 UTC m=+1100.971164139" watchObservedRunningTime="2026-10-30 18:56:16.900573943 +0000 UTC m=+1100.972398018" Oct 30 18:56:29.281316 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:56:29.281273 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:56:47.795865 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:47.795836 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:56:54.585729 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:54.585697 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:56:57.796472 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:56:57.796433 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:56:59.280062 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:56:59.280024 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:57:07.796576 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:07.796532 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:57:09.606083 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:09.606025 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" containerID="cri-o://ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb" gracePeriod=45 Oct 30 18:57:17.795740 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:17.795702 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:57:17.796165 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:17.795812 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:57:27.795803 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:27.795766 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:57:29.280174 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:57:29.280140 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:57:36.654848 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.654824 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5_a4b03a64-8755-404f-b3fb-ca2ed66b9e7d/main/0.log" Oct 30 18:57:36.657133 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.657117 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:57:36.772011 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.771979 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\") pod \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " Oct 30 18:57:36.772165 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.772032 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\" (UniqueName: \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\") pod \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\" (UID: \"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d\") " Oct 30 18:57:36.774369 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.774337 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw" pod "a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" (UID: "a4b03a64-8755-404f-b3fb-ca2ed66b9e7d"). InnerVolumeSpecName "kube-api-access-r6bpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:57:36.774460 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.774434 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs" pod "a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" (UID: "a4b03a64-8755-404f-b3fb-ca2ed66b9e7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:57:36.873944 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.873914 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:57:36.873944 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:36.873942 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6bpw\" (UniqueName: \"kubernetes.io/projected/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d-kube-api-access-r6bpw\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:57:37.147345 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147325 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5_a4b03a64-8755-404f-b3fb-ca2ed66b9e7d/main/0.log" Oct 30 18:57:37.147449 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147359 5869 generic.go:356] "Generic (PLEG): container finished" podID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerID="ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb" exitCode=1 Oct 30 18:57:37.147449 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147394 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" event={"ID":"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d","Type":"ContainerDied","Data":"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb"} Oct 30 18:57:37.147449 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147425 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" Oct 30 18:57:37.147449 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147446 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5" event={"ID":"a4b03a64-8755-404f-b3fb-ca2ed66b9e7d","Type":"ContainerDied","Data":"7b006b3cff3746b95db9b2973e46158e41114ace13ddfdd78533774a39d89362"} Oct 30 18:57:37.147583 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.147470 5869 scope.go:122] "RemoveContainer" containerID="ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb" Oct 30 18:57:37.156600 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.156576 5869 scope.go:122] "RemoveContainer" containerID="ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb" Oct 30 18:57:37.156847 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:57:37.156827 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb\": container with ID starting with ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb not found: ID does not exist" containerID="ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb" Oct 30 18:57:37.156922 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.156878 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb"} err="failed to get container status \"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb\": rpc error: code = NotFound desc = could not find container \"ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb\": container with ID starting with ceb91608f5223d20e695f0729558d5ae568c121731c43ef228ea2ee583122cdb not found: ID does not exist" Oct 30 18:57:37.307638 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.307607 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:57:37.354891 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:37.354865 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b6f2xp5"] Oct 30 18:57:38.040678 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:38.040641 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" path="/var/lib/kubelet/pods/a4b03a64-8755-404f-b3fb-ca2ed66b9e7d/volumes" Oct 30 18:57:46.251220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.251189 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:57:46.251679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.251543 5869 cpu_manager.go:414] "containerMap: removing container" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" Oct 30 18:57:46.251679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.251553 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" Oct 30 18:57:46.251679 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.251635 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4b03a64-8755-404f-b3fb-ca2ed66b9e7d" containerName="main" Oct 30 18:57:46.254738 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.254722 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.310360 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.310335 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:57:46.350436 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.350410 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.350569 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.350451 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpz86\" (UniqueName: \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.356220 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.356202 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-8g4pg\"" Oct 30 18:57:46.356299 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.356200 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\"" Oct 30 18:57:46.356299 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.356200 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-b4mrj\"" Oct 30 18:57:46.405192 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.405164 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\"" Oct 30 18:57:46.405328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.405208 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\"" Oct 30 18:57:46.451463 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.451431 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.451618 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.451471 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpz86\" (UniqueName: \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.454343 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.454311 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.507748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.507684 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpz86\" (UniqueName: \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.567696 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.567673 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:46.936765 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:46.936734 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:57:46.945207 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:57:46.945166 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00364523_928a_4ca1_8545_a930ec4f551d.slice/crio-381b9d2ab227d9384d1746c3b33ea00558b3476a93b43f0f8e86fdf4f536d868 WatchSource:0}: Error finding container 381b9d2ab227d9384d1746c3b33ea00558b3476a93b43f0f8e86fdf4f536d868: Status 404 returned error can't find the container with id 381b9d2ab227d9384d1746c3b33ea00558b3476a93b43f0f8e86fdf4f536d868 Oct 30 18:57:47.193048 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:47.192964 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" event={"ID":"00364523-928a-4ca1-8545-a930ec4f551d","Type":"ContainerStarted","Data":"1784c26b41bb7b6068410d06a40f593ec01a638f89ca5207543e8248bbf7f0fd"} Oct 30 18:57:47.193048 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:47.193016 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" event={"ID":"00364523-928a-4ca1-8545-a930ec4f551d","Type":"ContainerStarted","Data":"381b9d2ab227d9384d1746c3b33ea00558b3476a93b43f0f8e86fdf4f536d868"} Oct 30 18:57:47.193253 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:47.193084 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:57:47.309975 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:57:47.309928 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podStartSLOduration=1.3099148760000001 podStartE2EDuration="1.309914876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:47.308288514 +0000 UTC m=+1191.380112589" watchObservedRunningTime="2026-10-30 18:57:47.309914876 +0000 UTC m=+1191.381738951" Oct 30 18:57:59.280558 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:57:59.280523 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:58:18.214781 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:18.214752 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:58:29.280258 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:58:29.280223 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:58:29.369389 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:29.369358 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:58:38.197934 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:38.197896 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:58:44.389705 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:44.389663 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" containerID="cri-o://1784c26b41bb7b6068410d06a40f593ec01a638f89ca5207543e8248bbf7f0fd" gracePeriod=45 Oct 30 18:58:48.198599 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:48.198563 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:58:58.198406 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:58.198364 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:58:58.198885 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:58:58.198465 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:58:59.281987 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:58:59.281949 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:59:08.198676 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:08.198580 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:59:14.623435 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.623406 5869 generic.go:356] "Generic (PLEG): container finished" podID="00364523-928a-4ca1-8545-a930ec4f551d" containerID="1784c26b41bb7b6068410d06a40f593ec01a638f89ca5207543e8248bbf7f0fd" exitCode=0 Oct 30 18:59:14.623741 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.623450 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" event={"ID":"00364523-928a-4ca1-8545-a930ec4f551d","Type":"ContainerDied","Data":"1784c26b41bb7b6068410d06a40f593ec01a638f89ca5207543e8248bbf7f0fd"} Oct 30 18:59:14.640721 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.640697 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:59:14.722948 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.722925 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\" (UniqueName: \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\") pod \"00364523-928a-4ca1-8545-a930ec4f551d\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " Oct 30 18:59:14.723055 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.722994 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\" (UniqueName: \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\") pod \"00364523-928a-4ca1-8545-a930ec4f551d\" (UID: \"00364523-928a-4ca1-8545-a930ec4f551d\") " Oct 30 18:59:14.725213 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.725186 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs" pod "00364523-928a-4ca1-8545-a930ec4f551d" (UID: "00364523-928a-4ca1-8545-a930ec4f551d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:59:14.725318 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.725272 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86" pod "00364523-928a-4ca1-8545-a930ec4f551d" (UID: "00364523-928a-4ca1-8545-a930ec4f551d"). InnerVolumeSpecName "kube-api-access-qpz86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:59:14.823815 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.823788 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpz86\" (UniqueName: \"kubernetes.io/projected/00364523-928a-4ca1-8545-a930ec4f551d-kube-api-access-qpz86\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:14.823815 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:14.823813 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00364523-928a-4ca1-8545-a930ec4f551d-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:15.629543 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:15.629516 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" Oct 30 18:59:15.629958 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:15.629562 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj" event={"ID":"00364523-928a-4ca1-8545-a930ec4f551d","Type":"ContainerDied","Data":"381b9d2ab227d9384d1746c3b33ea00558b3476a93b43f0f8e86fdf4f536d868"} Oct 30 18:59:15.629958 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:15.629616 5869 scope.go:122] "RemoveContainer" containerID="1784c26b41bb7b6068410d06a40f593ec01a638f89ca5207543e8248bbf7f0fd" Oct 30 18:59:15.788584 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:15.788559 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:59:15.835743 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:15.835718 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-nj4hj"] Oct 30 18:59:16.040294 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:16.040263 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00364523-928a-4ca1-8545-a930ec4f551d" path="/var/lib/kubelet/pods/00364523-928a-4ca1-8545-a930ec4f551d/volumes" Oct 30 18:59:27.503344 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.503313 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 18:59:27.503723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.503693 5869 cpu_manager.go:414] "containerMap: removing container" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" Oct 30 18:59:27.503723 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.503707 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" Oct 30 18:59:27.503825 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.503815 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="00364523-928a-4ca1-8545-a930ec4f551d" containerName="main" Oct 30 18:59:27.506806 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.506790 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.564600 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.564559 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 18:59:27.576583 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.576558 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\"" Oct 30 18:59:27.576751 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.576625 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\"" Oct 30 18:59:27.610269 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.610246 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-px4cx\"" Oct 30 18:59:27.610269 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.610256 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\"" Oct 30 18:59:27.620511 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620485 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.620593 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620519 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.620593 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620541 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.620678 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620603 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.620712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620679 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.620712 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.620706 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bmv8\" (UniqueName: \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722171 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722146 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722183 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722205 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722226 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722249 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722276 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bmv8\" (UniqueName: \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.722622 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722595 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.723043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.722938 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.723043 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.723033 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.724753 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.724727 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.725193 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.725171 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.779229 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.779167 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bmv8\" (UniqueName: \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\") pod \"fc-concurrency-test-kserve-5485c7876d-v6thm\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:27.820253 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:27.820215 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:28.187546 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.187520 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 18:59:28.187684 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:59:28.187634 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ef2b00_4b46_4737_8dc2_bbd3c1f65fa5.slice/crio-c62c4c9f9ce0fd3a974ab054602f25533e4d36f21e5214a3bb302b574182039e WatchSource:0}: Error finding container c62c4c9f9ce0fd3a974ab054602f25533e4d36f21e5214a3bb302b574182039e: Status 404 returned error can't find the container with id c62c4c9f9ce0fd3a974ab054602f25533e4d36f21e5214a3bb302b574182039e Oct 30 18:59:28.189617 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.189600 5869 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:59:28.196455 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.196428 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 18:59:28.200975 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.200956 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.294661 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.294620 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-rgwnr\"" Oct 30 18:59:28.295754 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.295734 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 18:59:28.328256 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.328233 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mncb\" (UniqueName: \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.328369 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.328292 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.429502 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.429472 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.429662 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.429538 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mncb\" (UniqueName: \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.432135 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.432080 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.488883 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.488828 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mncb\" (UniqueName: \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.513759 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.513732 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:28.695268 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.695232 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" event={"ID":"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5","Type":"ContainerStarted","Data":"c62c4c9f9ce0fd3a974ab054602f25533e4d36f21e5214a3bb302b574182039e"} Oct 30 18:59:28.880682 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:28.880657 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 18:59:28.883440 ip-10-0-128-38 kubenswrapper[5869]: W1030 18:59:28.883403 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fabb747_56ad_43a1_815d_24b26fbe6c2f.slice/crio-028ccd2b4e10eab2c7d941b23cfc0c8a7b8cb466b99e9a6cc5dec6aed5246be7 WatchSource:0}: Error finding container 028ccd2b4e10eab2c7d941b23cfc0c8a7b8cb466b99e9a6cc5dec6aed5246be7: Status 404 returned error can't find the container with id 028ccd2b4e10eab2c7d941b23cfc0c8a7b8cb466b99e9a6cc5dec6aed5246be7 Oct 30 18:59:29.291306 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:59:29.291224 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 18:59:29.705575 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:29.705528 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" event={"ID":"0fabb747-56ad-43a1-815d-24b26fbe6c2f","Type":"ContainerStarted","Data":"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98"} Oct 30 18:59:29.705967 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:29.705597 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" event={"ID":"0fabb747-56ad-43a1-815d-24b26fbe6c2f","Type":"ContainerStarted","Data":"028ccd2b4e10eab2c7d941b23cfc0c8a7b8cb466b99e9a6cc5dec6aed5246be7"} Oct 30 18:59:29.705967 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:29.705624 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 18:59:29.816296 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:29.816243 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podStartSLOduration=1.8162284629999998 podStartE2EDuration="1.816228463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:29.815376883 +0000 UTC m=+1293.887200960" watchObservedRunningTime="2026-10-30 18:59:29.816228463 +0000 UTC m=+1293.888052539" Oct 30 18:59:30.711748 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:30.711709 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" event={"ID":"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5","Type":"ContainerStarted","Data":"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e"} Oct 30 18:59:30.820328 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:30.820297 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:30.822834 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:30.822771 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" podStartSLOduration=1.9144895119999998 podStartE2EDuration="3.822755813s" totalImagesPullingTime="1.908266301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:30.822277842 +0000 UTC m=+1294.894101918" watchObservedRunningTime="2026-10-30 18:59:30.822755813 +0000 UTC m=+1294.894579890" Oct 30 18:59:37.820877 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:37.820828 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:37.840382 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:37.840355 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:38.782568 ip-10-0-128-38 kubenswrapper[5869]: I1030 18:59:38.782540 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 18:59:59.279527 ip-10-0-128-38 kubenswrapper[5869]: E1030 18:59:59.279489 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:00:00.714010 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:00.713979 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 19:00:14.013215 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.013181 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 19:00:14.022714 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:00:14.022674 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" cmd=["/bin/sleep","15"] Oct 30 19:00:14.023194 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.022732 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:00:14.023194 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:00:14.022746 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerName="main" containerID="cri-o://5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" Oct 30 19:00:14.023194 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.022799 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerName="main" containerID="cri-o://5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" gracePeriod=60 Oct 30 19:00:14.030352 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.030330 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 19:00:14.373140 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.373117 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 19:00:14.401257 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401232 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401360 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401265 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401360 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401302 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\" (UniqueName: \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401360 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401326 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401529 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401393 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401529 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401440 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\" (UniqueName: \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\") pod \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\" (UID: \"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5\") " Oct 30 19:00:14.401529 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401485 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:00:14.401686 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401526 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:00:14.401686 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401639 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:00:14.401767 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401746 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.401767 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401761 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.401873 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.401772 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.403659 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.403639 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:00:14.403742 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.403703 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "kube-api-access-2bmv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:00:14.403742 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.403702 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm" pod "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" (UID: "94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:00:14.502694 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.502671 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bmv8\" (UniqueName: \"kubernetes.io/projected/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-kube-api-access-2bmv8\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.502694 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.502696 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.502853 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.502709 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:14.947452 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.947414 5869 generic.go:356] "Generic (PLEG): container finished" podID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerID="5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" exitCode=0 Oct 30 19:00:14.947628 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.947475 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" Oct 30 19:00:14.947628 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.947492 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" event={"ID":"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5","Type":"ContainerDied","Data":"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e"} Oct 30 19:00:14.947628 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.947541 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm" event={"ID":"94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5","Type":"ContainerDied","Data":"c62c4c9f9ce0fd3a974ab054602f25533e4d36f21e5214a3bb302b574182039e"} Oct 30 19:00:14.947628 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.947565 5869 scope.go:122] "RemoveContainer" containerID="5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" Oct 30 19:00:14.956746 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.956727 5869 scope.go:122] "RemoveContainer" containerID="5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" Oct 30 19:00:14.957034 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:00:14.957015 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e\": container with ID starting with 5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e not found: ID does not exist" containerID="5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e" Oct 30 19:00:14.957106 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:14.957046 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e"} err="failed to get container status \"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e\": rpc error: code = NotFound desc = could not find container \"5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e\": container with ID starting with 5dc94eb2e5b912192446858fbf37343751058a9fa4121b108d333f2bb45a816e not found: ID does not exist" Oct 30 19:00:15.106442 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:15.106414 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 19:00:15.153961 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:15.153938 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-v6thm"] Oct 30 19:00:16.041589 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:16.041559 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" path="/var/lib/kubelet/pods/94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5/volumes" Oct 30 19:00:20.714011 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:20.713976 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:00:29.049850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:29.049796 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" containerID="cri-o://81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98" gracePeriod=45 Oct 30 19:00:29.280491 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:00:29.280456 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:00:30.713326 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:30.713292 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:00:40.714085 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:40.714050 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:00:40.714549 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:40.714194 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 19:00:50.713175 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:50.713136 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:00:59.279647 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:00:59.279608 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:00:59.302819 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.302793 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 19:00:59.473293 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.473267 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\") pod \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " Oct 30 19:00:59.473444 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.473358 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\" (UniqueName: \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\") pod \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\" (UID: \"0fabb747-56ad-43a1-815d-24b26fbe6c2f\") " Oct 30 19:00:59.475449 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.475423 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb" pod "0fabb747-56ad-43a1-815d-24b26fbe6c2f" (UID: "0fabb747-56ad-43a1-815d-24b26fbe6c2f"). InnerVolumeSpecName "kube-api-access-4mncb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:00:59.475551 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.475447 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs" pod "0fabb747-56ad-43a1-815d-24b26fbe6c2f" (UID: "0fabb747-56ad-43a1-815d-24b26fbe6c2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:00:59.574271 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.574248 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mncb\" (UniqueName: \"kubernetes.io/projected/0fabb747-56ad-43a1-815d-24b26fbe6c2f-kube-api-access-4mncb\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:00:59.574271 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:00:59.574268 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fabb747-56ad-43a1-815d-24b26fbe6c2f-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:00.140752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.140718 5869 generic.go:356] "Generic (PLEG): container finished" podID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerID="81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98" exitCode=0 Oct 30 19:01:00.140912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.140772 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" Oct 30 19:01:00.140912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.140776 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" event={"ID":"0fabb747-56ad-43a1-815d-24b26fbe6c2f","Type":"ContainerDied","Data":"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98"} Oct 30 19:01:00.140912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.140887 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq" event={"ID":"0fabb747-56ad-43a1-815d-24b26fbe6c2f","Type":"ContainerDied","Data":"028ccd2b4e10eab2c7d941b23cfc0c8a7b8cb466b99e9a6cc5dec6aed5246be7"} Oct 30 19:01:00.140912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.140909 5869 scope.go:122] "RemoveContainer" containerID="81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98" Oct 30 19:01:00.148978 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.148962 5869 scope.go:122] "RemoveContainer" containerID="81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98" Oct 30 19:01:00.149307 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:00.149285 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98\": container with ID starting with 81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98 not found: ID does not exist" containerID="81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98" Oct 30 19:01:00.149388 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.149314 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98"} err="failed to get container status \"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98\": rpc error: code = NotFound desc = could not find container \"81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98\": container with ID starting with 81494d26e84a88c1d8c63bce9b8e3d68b9d220f5223405ffe25f6b1466c38e98 not found: ID does not exist" Oct 30 19:01:00.315082 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.315017 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 19:01:00.362315 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:00.362293 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-8c4xq"] Oct 30 19:01:02.040872 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:02.040838 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" path="/var/lib/kubelet/pods/0fabb747-56ad-43a1-815d-24b26fbe6c2f/volumes" Oct 30 19:01:07.742343 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742307 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742667 5869 cpu_manager.go:414] "containerMap: removing container" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerName="main" Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742680 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerName="main" Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742706 5869 cpu_manager.go:414] "containerMap: removing container" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742711 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742775 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fabb747-56ad-43a1-815d-24b26fbe6c2f" containerName="main" Oct 30 19:01:07.742792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.742791 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="94ef2b00-4b46-4737-8dc2-bbd3c1f65fa5" containerName="main" Oct 30 19:01:07.747384 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.747366 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.839136 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.839080 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:07.842593 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842571 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.842682 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842605 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6f84\" (UniqueName: \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.842682 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842632 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.842817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842795 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.842849 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842837 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.842905 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.842893 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.849834 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.849805 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\"" Oct 30 19:01:07.849834 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.849828 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\"" Oct 30 19:01:07.849980 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.849861 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-9w5px\"" Oct 30 19:01:07.849980 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.849871 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\"" Oct 30 19:01:07.943311 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943282 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943459 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943322 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943459 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943350 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943532 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943464 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943532 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943515 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6f84\" (UniqueName: \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943611 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943547 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943713 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943694 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943821 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943790 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.943942 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.943849 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.945749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.945726 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:07.945954 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:07.945936 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:08.005611 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:08.005563 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6f84\" (UniqueName: \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:08.059351 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:08.059331 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:08.436537 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:08.436514 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:08.438786 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:01:08.438754 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4924011a_f3f6_49e9_8997_5cc8e9f77d6f.slice/crio-3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2 WatchSource:0}: Error finding container 3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2: Status 404 returned error can't find the container with id 3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2 Oct 30 19:01:09.183337 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:09.183304 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" event={"ID":"4924011a-f3f6-49e9-8997-5cc8e9f77d6f","Type":"ContainerStarted","Data":"e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb"} Oct 30 19:01:09.183736 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:09.183391 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" event={"ID":"4924011a-f3f6-49e9-8997-5cc8e9f77d6f","Type":"ContainerStarted","Data":"3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2"} Oct 30 19:01:09.300467 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:09.300414 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" podStartSLOduration=2.300398922 podStartE2EDuration="2.300398922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:09.299983892 +0000 UTC m=+1393.371807972" watchObservedRunningTime="2026-10-30 19:01:09.300398922 +0000 UTC m=+1393.372223000" Oct 30 19:01:10.060155 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:10.060123 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:11.535515 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.535482 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:11.539087 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.539069 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.575686 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.575661 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.575814 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.575701 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.575884 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.575861 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd6wr\" (UniqueName: \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.575974 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.575959 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.576021 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.575982 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.576021 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.576001 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.592269 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.592246 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"default-dockercfg-7vfjs\"" Oct 30 19:01:11.639618 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.639595 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\"" Oct 30 19:01:11.639737 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.639723 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\"" Oct 30 19:01:11.640902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.640883 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:11.676505 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676483 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676601 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676507 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676601 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676526 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676601 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676549 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676601 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676569 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676756 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676612 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd6wr\" (UniqueName: \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676920 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676899 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.676972 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676952 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.677013 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.676984 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.678430 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.678414 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\"" Oct 30 19:01:11.678775 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.678750 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.688875 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.688853 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.732228 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.732205 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd6wr\" (UniqueName: \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:11.852042 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:11.851950 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:12.370227 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:12.370140 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:13.248930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:13.248890 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" event={"ID":"6c5534bf-0b1a-4c2f-840c-7824bc3e8859","Type":"ContainerStarted","Data":"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9"} Oct 30 19:01:13.249445 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:13.248944 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" event={"ID":"6c5534bf-0b1a-4c2f-840c-7824bc3e8859","Type":"ContainerStarted","Data":"a799885cdaeb3b98792d8416f3be5aa6dd3ad0f569f11a68fcb150ccddcd93dc"} Oct 30 19:01:13.372121 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:13.368800 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" podStartSLOduration=2.368779246 podStartE2EDuration="2.368779246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:13.367488384 +0000 UTC m=+1397.439312459" watchObservedRunningTime="2026-10-30 19:01:13.368779246 +0000 UTC m=+1397.440603323" Oct 30 19:01:13.852060 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:13.852019 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:18.059710 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:18.059677 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:18.072246 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:18.072218 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:18.281426 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:18.281403 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:20.284650 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.284606 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:20.294040 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:20.294004 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" cmd=["/bin/sleep","15"] Oct 30 19:01:20.294202 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.294063 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:01:20.294202 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:20.294086 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerName="main" containerID="cri-o://d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" Oct 30 19:01:20.294202 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.294143 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerName="main" containerID="cri-o://d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" gracePeriod=60 Oct 30 19:01:20.532932 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.532908 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:20.572770 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572707 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.572886 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572784 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.572886 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572855 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\" (UniqueName: \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.573007 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572887 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.573007 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572927 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.573007 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572934 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:20.573007 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.572977 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\") pod \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\" (UID: \"6c5534bf-0b1a-4c2f-840c-7824bc3e8859\") " Oct 30 19:01:20.573229 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.573021 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:20.573229 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.573171 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:20.573335 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.573318 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:20.573387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.573341 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:20.573387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.573355 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:20.575204 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.575179 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:20.575297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.575216 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:01:20.575297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.575237 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr" pod "6c5534bf-0b1a-4c2f-840c-7824bc3e8859" (UID: "6c5534bf-0b1a-4c2f-840c-7824bc3e8859"). InnerVolumeSpecName "kube-api-access-bd6wr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:01:20.674451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.674425 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd6wr\" (UniqueName: \"kubernetes.io/projected/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-kube-api-access-bd6wr\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:20.674451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.674448 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:20.674451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:20.674458 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c5534bf-0b1a-4c2f-840c-7824bc3e8859-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:21.285975 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.285944 5869 generic.go:356] "Generic (PLEG): container finished" podID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerID="d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" exitCode=0 Oct 30 19:01:21.286500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.285995 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" Oct 30 19:01:21.286500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.286035 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" event={"ID":"6c5534bf-0b1a-4c2f-840c-7824bc3e8859","Type":"ContainerDied","Data":"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9"} Oct 30 19:01:21.286500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.286123 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc" event={"ID":"6c5534bf-0b1a-4c2f-840c-7824bc3e8859","Type":"ContainerDied","Data":"a799885cdaeb3b98792d8416f3be5aa6dd3ad0f569f11a68fcb150ccddcd93dc"} Oct 30 19:01:21.286500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.286158 5869 scope.go:122] "RemoveContainer" containerID="d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" Oct 30 19:01:21.302235 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.302083 5869 scope.go:122] "RemoveContainer" containerID="d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" Oct 30 19:01:21.302381 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:21.302361 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9\": container with ID starting with d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9 not found: ID does not exist" containerID="d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9" Oct 30 19:01:21.302423 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.302388 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9"} err="failed to get container status \"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9\": rpc error: code = NotFound desc = could not find container \"d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9\": container with ID starting with d5ac403a7d99af4b208643015c6500ea33918affbf768cca59da809d4945a7f9 not found: ID does not exist" Oct 30 19:01:21.449106 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.449065 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:21.456755 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:21.456731 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-4m5qc"] Oct 30 19:01:22.040392 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:22.040360 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" path="/var/lib/kubelet/pods/6c5534bf-0b1a-4c2f-840c-7824bc3e8859/volumes" Oct 30 19:01:28.964567 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:28.964532 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:28.964960 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:28.964923 5869 cpu_manager.go:414] "containerMap: removing container" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerName="main" Oct 30 19:01:28.964960 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:28.964936 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerName="main" Oct 30 19:01:28.965033 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:28.965020 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c5534bf-0b1a-4c2f-840c-7824bc3e8859" containerName="main" Oct 30 19:01:28.969326 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:28.969310 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.023531 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.023505 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:29.049303 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049277 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt4l2\" (UniqueName: \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.049415 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049311 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.049415 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049397 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.049500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049429 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.049500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049458 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.049580 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.049526 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.069754 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.069733 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\"" Oct 30 19:01:29.069835 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.069754 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\"" Oct 30 19:01:29.069835 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.069793 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-5hm6b\"" Oct 30 19:01:29.071784 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.071769 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\"" Oct 30 19:01:29.150432 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150410 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt4l2\" (UniqueName: \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150554 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150444 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150554 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150490 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150554 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150537 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150689 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150581 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150689 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150611 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.150928 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150905 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.151010 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150910 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.151010 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.150967 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.152791 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.152770 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.153088 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.153071 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.206495 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.206464 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt4l2\" (UniqueName: \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.280235 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:29.280154 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:01:29.282693 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.282669 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:29.649698 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:29.649673 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:29.652025 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:01:29.651979 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31f881d_d935_407b_b83c_67d153c418ab.slice/crio-2c62a48c4f0534d2dafbd0fc88350c062049e05f24bb3fb04eda35c72c679a07 WatchSource:0}: Error finding container 2c62a48c4f0534d2dafbd0fc88350c062049e05f24bb3fb04eda35c72c679a07: Status 404 returned error can't find the container with id 2c62a48c4f0534d2dafbd0fc88350c062049e05f24bb3fb04eda35c72c679a07 Oct 30 19:01:30.328108 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:30.328071 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" event={"ID":"c31f881d-d935-407b-b83c-67d153c418ab","Type":"ContainerStarted","Data":"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481"} Oct 30 19:01:30.328506 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:30.328139 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" event={"ID":"c31f881d-d935-407b-b83c-67d153c418ab","Type":"ContainerStarted","Data":"2c62a48c4f0534d2dafbd0fc88350c062049e05f24bb3fb04eda35c72c679a07"} Oct 30 19:01:30.441593 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:30.441540 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" podStartSLOduration=2.441526932 podStartE2EDuration="2.441526932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:30.440396362 +0000 UTC m=+1414.512220438" watchObservedRunningTime="2026-10-30 19:01:30.441526932 +0000 UTC m=+1414.513351008" Oct 30 19:01:31.283856 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:31.283822 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:32.649517 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:32.649488 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:33.354060 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:33.354019 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" cmd=["/bin/sleep","15"] Oct 30 19:01:33.354270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.354088 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:01:33.354270 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:33.354127 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" podUID="c31f881d-d935-407b-b83c-67d153c418ab" containerName="main" containerID="cri-o://bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" Oct 30 19:01:33.354270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.354170 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" podUID="c31f881d-d935-407b-b83c-67d153c418ab" containerName="main" containerID="cri-o://bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" gracePeriod=60 Oct 30 19:01:33.619409 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.619341 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:33.693912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.693885 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.693993 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\" (UniqueName: \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694029 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694078 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694212 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\" (UniqueName: \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694264 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\") pod \"c31f881d-d935-407b-b83c-67d153c418ab\" (UID: \"c31f881d-d935-407b-b83c-67d153c418ab\") " Oct 30 19:01:33.694552 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694316 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:33.694552 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694363 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:33.694552 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694515 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:33.694656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694592 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:33.694656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694610 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:33.694656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.694625 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:33.696400 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.696375 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:33.696400 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.696381 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:01:33.696543 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.696454 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2" pod "c31f881d-d935-407b-b83c-67d153c418ab" (UID: "c31f881d-d935-407b-b83c-67d153c418ab"). InnerVolumeSpecName "kube-api-access-pt4l2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:01:33.796006 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.795984 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31f881d-d935-407b-b83c-67d153c418ab-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:33.796006 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.796004 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt4l2\" (UniqueName: \"kubernetes.io/projected/c31f881d-d935-407b-b83c-67d153c418ab-kube-api-access-pt4l2\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:33.796178 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:33.796014 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31f881d-d935-407b-b83c-67d153c418ab-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:34.351768 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.351695 5869 generic.go:356] "Generic (PLEG): container finished" podID="c31f881d-d935-407b-b83c-67d153c418ab" containerID="bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" exitCode=0 Oct 30 19:01:34.351768 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.351744 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" Oct 30 19:01:34.351943 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.351779 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" event={"ID":"c31f881d-d935-407b-b83c-67d153c418ab","Type":"ContainerDied","Data":"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481"} Oct 30 19:01:34.351943 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.351832 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl" event={"ID":"c31f881d-d935-407b-b83c-67d153c418ab","Type":"ContainerDied","Data":"2c62a48c4f0534d2dafbd0fc88350c062049e05f24bb3fb04eda35c72c679a07"} Oct 30 19:01:34.351943 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.351854 5869 scope.go:122] "RemoveContainer" containerID="bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" Oct 30 19:01:34.361185 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.360951 5869 scope.go:122] "RemoveContainer" containerID="bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" Oct 30 19:01:34.361289 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:34.361270 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481\": container with ID starting with bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481 not found: ID does not exist" containerID="bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481" Oct 30 19:01:34.361333 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.361302 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481"} err="failed to get container status \"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481\": rpc error: code = NotFound desc = could not find container \"bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481\": container with ID starting with bfd139be223b965830ef4189f45f08f13926f14c879c18a361777f5622f43481 not found: ID does not exist" Oct 30 19:01:34.511108 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.511058 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:34.558211 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:34.558184 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-8pqrl"] Oct 30 19:01:36.042716 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:36.042687 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c31f881d-d935-407b-b83c-67d153c418ab" path="/var/lib/kubelet/pods/c31f881d-d935-407b-b83c-67d153c418ab/volumes" Oct 30 19:01:49.915378 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.915348 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:01:49.915737 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.915717 5869 cpu_manager.go:414] "containerMap: removing container" podUID="c31f881d-d935-407b-b83c-67d153c418ab" containerName="main" Oct 30 19:01:49.915737 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.915728 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c31f881d-d935-407b-b83c-67d153c418ab" containerName="main" Oct 30 19:01:49.915819 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.915809 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="c31f881d-d935-407b-b83c-67d153c418ab" containerName="main" Oct 30 19:01:49.918650 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.918631 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:49.999667 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:49.999635 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:01:50.018597 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.018579 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-sdptj\"" Oct 30 19:01:50.018699 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.018597 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-fwgqr\"" Oct 30 19:01:50.018739 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.018717 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\"" Oct 30 19:01:50.018890 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.018873 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\"" Oct 30 19:01:50.018924 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.018893 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\"" Oct 30 19:01:50.033344 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.033324 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n86pm\" (UniqueName: \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.033401 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.033365 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.134238 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.134205 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n86pm\" (UniqueName: \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.134399 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.134251 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.136592 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.136566 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.192371 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.192343 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n86pm\" (UniqueName: \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.230281 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.230256 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:50.596449 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:50.596299 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:01:50.599024 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:01:50.598998 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8010ebda_31a8_4732_88e0_758c754d2cda.slice/crio-7904e49607ad02478cab723be7cf6c9140e5717ebb8941293ff4546f1dc9de7c WatchSource:0}: Error finding container 7904e49607ad02478cab723be7cf6c9140e5717ebb8941293ff4546f1dc9de7c: Status 404 returned error can't find the container with id 7904e49607ad02478cab723be7cf6c9140e5717ebb8941293ff4546f1dc9de7c Oct 30 19:01:51.431800 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:51.431762 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" event={"ID":"8010ebda-31a8-4732-88e0-758c754d2cda","Type":"ContainerStarted","Data":"9f2e77664fdb11dc3eef3575bbd976f7c2af07717be8b56a1524b10951fb5f3b"} Oct 30 19:01:51.432227 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:51.431826 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" event={"ID":"8010ebda-31a8-4732-88e0-758c754d2cda","Type":"ContainerStarted","Data":"7904e49607ad02478cab723be7cf6c9140e5717ebb8941293ff4546f1dc9de7c"} Oct 30 19:01:51.432227 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:51.431860 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:01:51.543103 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:51.543042 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podStartSLOduration=2.543030637 podStartE2EDuration="2.543030637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:51.542724273 +0000 UTC m=+1435.614548348" watchObservedRunningTime="2026-10-30 19:01:51.543030637 +0000 UTC m=+1435.614854713" Oct 30 19:01:56.108361 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.108326 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:56.118208 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:56.117882 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb" cmd=["/bin/sleep","15"] Oct 30 19:01:56.118208 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.117948 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:01:56.118208 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:56.117966 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" containerID="cri-o://e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb" Oct 30 19:01:56.118208 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.118025 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" containerID="cri-o://e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb" gracePeriod=60 Oct 30 19:01:56.270729 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.270695 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": dial tcp 10.133.0.36:8000: connect: connection refused" Oct 30 19:01:56.454701 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.454672 5869 generic.go:356] "Generic (PLEG): container finished" podID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerID="e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb" exitCode=0 Oct 30 19:01:56.454850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.454741 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" event={"ID":"4924011a-f3f6-49e9-8997-5cc8e9f77d6f","Type":"ContainerDied","Data":"e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb"} Oct 30 19:01:56.454850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.454815 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" event={"ID":"4924011a-f3f6-49e9-8997-5cc8e9f77d6f","Type":"ContainerDied","Data":"3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2"} Oct 30 19:01:56.454850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.454831 5869 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3583f3763721fd915e395cea325b19930af5a390748562232383ad4ff74894f2" Oct 30 19:01:56.470421 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.470402 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:56.591368 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591341 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591511 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591376 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591511 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591408 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591511 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591453 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591511 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591509 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\" (UniqueName: \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591724 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591576 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\" (UniqueName: \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\") pod \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\" (UID: \"4924011a-f3f6-49e9-8997-5cc8e9f77d6f\") " Oct 30 19:01:56.591724 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591652 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:56.591724 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591641 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:56.591886 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591722 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:56.591886 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591853 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:56.591886 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591883 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:56.592024 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.591893 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:56.593847 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.593824 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:56.593847 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.593835 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:01:56.594032 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.593872 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84" pod "4924011a-f3f6-49e9-8997-5cc8e9f77d6f" (UID: "4924011a-f3f6-49e9-8997-5cc8e9f77d6f"). InnerVolumeSpecName "kube-api-access-w6f84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:01:56.693397 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.693363 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:56.693535 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.693400 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6f84\" (UniqueName: \"kubernetes.io/projected/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-kube-api-access-w6f84\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:56.693535 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:56.693417 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4924011a-f3f6-49e9-8997-5cc8e9f77d6f-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:57.459985 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:57.459952 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh" Oct 30 19:01:57.617403 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:57.617378 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:57.664446 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:57.664414 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-77765cklh"] Oct 30 19:01:58.040781 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:01:58.040753 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" path="/var/lib/kubelet/pods/4924011a-f3f6-49e9-8997-5cc8e9f77d6f/volumes" Oct 30 19:01:59.282748 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:01:59.282711 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:02:22.438319 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:22.438291 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:02:29.279988 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:02:29.279940 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:02:52.478517 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.478481 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:02:52.479282 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.479250 5869 cpu_manager.go:414] "containerMap: removing container" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" Oct 30 19:02:52.479282 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.479275 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" Oct 30 19:02:52.479490 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.479396 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="4924011a-f3f6-49e9-8997-5cc8e9f77d6f" containerName="main" Oct 30 19:02:52.483462 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.483440 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.535389 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.535364 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\"" Oct 30 19:02:52.535389 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.535367 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\"" Oct 30 19:02:52.582155 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.582125 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-j2j9z\"" Oct 30 19:02:52.582155 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.582137 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-xgflg\"" Oct 30 19:02:52.582349 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.582194 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\"" Oct 30 19:02:52.583608 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.583590 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:02:52.642335 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.642302 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwnw\" (UniqueName: \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.642493 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.642357 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.743143 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.743044 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwnw\" (UniqueName: \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.743143 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.743082 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.746105 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.746056 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:52.799248 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:52.799224 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwnw\" (UniqueName: \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:53.096407 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.096316 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:53.467930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.467883 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:02:53.470964 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:02:53.470934 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65aa1e53_573c_4e63_9291_c707f5d984f6.slice/crio-7525929b620c8b9b6c9c079603c6c0b16a45e82c361526b4513f287f19c8c4d5 WatchSource:0}: Error finding container 7525929b620c8b9b6c9c079603c6c0b16a45e82c361526b4513f287f19c8c4d5: Status 404 returned error can't find the container with id 7525929b620c8b9b6c9c079603c6c0b16a45e82c361526b4513f287f19c8c4d5 Oct 30 19:02:53.721992 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.721919 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" event={"ID":"65aa1e53-573c-4e63-9291-c707f5d984f6","Type":"ContainerStarted","Data":"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99"} Oct 30 19:02:53.721992 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.721974 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" event={"ID":"65aa1e53-573c-4e63-9291-c707f5d984f6","Type":"ContainerStarted","Data":"7525929b620c8b9b6c9c079603c6c0b16a45e82c361526b4513f287f19c8c4d5"} Oct 30 19:02:53.722422 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.722010 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:02:53.833228 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:02:53.833185 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podStartSLOduration=1.833171213 podStartE2EDuration="1.833171213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:53.832389854 +0000 UTC m=+1497.904213930" watchObservedRunningTime="2026-10-30 19:02:53.833171213 +0000 UTC m=+1497.904995290" Oct 30 19:02:59.279806 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:02:59.279772 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:03:24.728839 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:03:24.728807 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:03:29.279578 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:03:29.279538 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:03:35.991564 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:03:35.991527 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:03:44.728326 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:03:44.728282 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:03:51.011831 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:03:51.011792 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" containerID="cri-o://8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99" gracePeriod=45 Oct 30 19:03:54.728564 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:03:54.728527 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:03:59.280501 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:03:59.280456 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:04:00.087435 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:00.087405 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:04:02.438081 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:02.438036 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:04.728109 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:04.728057 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:04.728545 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:04.728191 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:04:12.438020 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:12.437984 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:14.728229 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:14.728184 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:15.110257 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:15.110160 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" containerID="cri-o://9f2e77664fdb11dc3eef3575bbd976f7c2af07717be8b56a1524b10951fb5f3b" gracePeriod=45 Oct 30 19:04:21.260656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.260625 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:04:21.357648 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.357590 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\") pod \"65aa1e53-573c-4e63-9291-c707f5d984f6\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " Oct 30 19:04:21.357772 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.357728 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\" (UniqueName: \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\") pod \"65aa1e53-573c-4e63-9291-c707f5d984f6\" (UID: \"65aa1e53-573c-4e63-9291-c707f5d984f6\") " Oct 30 19:04:21.359788 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.359765 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs" pod "65aa1e53-573c-4e63-9291-c707f5d984f6" (UID: "65aa1e53-573c-4e63-9291-c707f5d984f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:04:21.359903 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.359865 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw" pod "65aa1e53-573c-4e63-9291-c707f5d984f6" (UID: "65aa1e53-573c-4e63-9291-c707f5d984f6"). InnerVolumeSpecName "kube-api-access-rtwnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:04:21.458416 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.458390 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtwnw\" (UniqueName: \"kubernetes.io/projected/65aa1e53-573c-4e63-9291-c707f5d984f6-kube-api-access-rtwnw\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:04:21.458416 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:21.458415 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65aa1e53-573c-4e63-9291-c707f5d984f6-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:04:22.115610 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.115578 5869 generic.go:356] "Generic (PLEG): container finished" podID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerID="8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99" exitCode=0 Oct 30 19:04:22.115752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.115625 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" Oct 30 19:04:22.115752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.115660 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" event={"ID":"65aa1e53-573c-4e63-9291-c707f5d984f6","Type":"ContainerDied","Data":"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99"} Oct 30 19:04:22.115752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.115708 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv" event={"ID":"65aa1e53-573c-4e63-9291-c707f5d984f6","Type":"ContainerDied","Data":"7525929b620c8b9b6c9c079603c6c0b16a45e82c361526b4513f287f19c8c4d5"} Oct 30 19:04:22.115752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.115730 5869 scope.go:122] "RemoveContainer" containerID="8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99" Oct 30 19:04:22.123978 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.123960 5869 scope.go:122] "RemoveContainer" containerID="8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99" Oct 30 19:04:22.124276 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:04:22.124258 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99\": container with ID starting with 8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99 not found: ID does not exist" containerID="8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99" Oct 30 19:04:22.124330 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.124286 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99"} err="failed to get container status \"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99\": rpc error: code = NotFound desc = could not find container \"8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99\": container with ID starting with 8720cb9dd94738aa2ed265a165c5327ea655f06aeee624d9cc4e83347d0c6e99 not found: ID does not exist" Oct 30 19:04:22.273732 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.273707 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:04:22.321159 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.321089 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588c9kvjv"] Oct 30 19:04:22.438258 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.438231 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:22.438364 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:22.438316 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:04:24.040521 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:24.040492 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" path="/var/lib/kubelet/pods/65aa1e53-573c-4e63-9291-c707f5d984f6/volumes" Oct 30 19:04:29.279708 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:04:29.279669 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:04:32.438696 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:32.438657 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:34.952324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:34.952292 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:04:34.952747 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:34.952731 5869 cpu_manager.go:414] "containerMap: removing container" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" Oct 30 19:04:34.952793 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:34.952748 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" Oct 30 19:04:34.952827 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:34.952823 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="65aa1e53-573c-4e63-9291-c707f5d984f6" containerName="main" Oct 30 19:04:34.957418 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:34.957398 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.011955 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.011926 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:04:35.029436 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.029408 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\"" Oct 30 19:04:35.029568 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.029442 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\"" Oct 30 19:04:35.057901 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.057876 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-kbdpg\"" Oct 30 19:04:35.058014 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.057917 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\"" Oct 30 19:04:35.058014 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.057949 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-sjwbl\"" Oct 30 19:04:35.078445 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.078417 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlc67\" (UniqueName: \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.078544 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.078457 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.179610 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.179580 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc67\" (UniqueName: \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.179775 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.179616 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.182378 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.182352 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.239751 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.239695 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc67\" (UniqueName: \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.269775 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.269749 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:35.633664 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:35.633610 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:04:35.635906 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:04:35.635881 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bdd79e_0d65_4879_9434_4a76087d2bc0.slice/crio-d72e2a7fdf9f9f8027d50444d3a63badbdc82d4934e1d05e0d0a29759caadaa2 WatchSource:0}: Error finding container d72e2a7fdf9f9f8027d50444d3a63badbdc82d4934e1d05e0d0a29759caadaa2: Status 404 returned error can't find the container with id d72e2a7fdf9f9f8027d50444d3a63badbdc82d4934e1d05e0d0a29759caadaa2 Oct 30 19:04:36.180406 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:36.180375 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" event={"ID":"78bdd79e-0d65-4879-9434-4a76087d2bc0","Type":"ContainerStarted","Data":"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1"} Oct 30 19:04:36.180843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:36.180421 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" event={"ID":"78bdd79e-0d65-4879-9434-4a76087d2bc0","Type":"ContainerStarted","Data":"d72e2a7fdf9f9f8027d50444d3a63badbdc82d4934e1d05e0d0a29759caadaa2"} Oct 30 19:04:36.180843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:36.180514 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:04:36.291149 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:36.291076 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podStartSLOduration=2.291065126 podStartE2EDuration="2.291065126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:36.290694813 +0000 UTC m=+1600.362518890" watchObservedRunningTime="2026-10-30 19:04:36.291065126 +0000 UTC m=+1600.362889201" Oct 30 19:04:42.438535 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:42.438487 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:04:45.241078 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.241050 5869 generic.go:356] "Generic (PLEG): container finished" podID="8010ebda-31a8-4732-88e0-758c754d2cda" containerID="9f2e77664fdb11dc3eef3575bbd976f7c2af07717be8b56a1524b10951fb5f3b" exitCode=0 Oct 30 19:04:45.241405 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.241136 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" event={"ID":"8010ebda-31a8-4732-88e0-758c754d2cda","Type":"ContainerDied","Data":"9f2e77664fdb11dc3eef3575bbd976f7c2af07717be8b56a1524b10951fb5f3b"} Oct 30 19:04:45.363984 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.363960 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:04:45.479157 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.479134 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\" (UniqueName: \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\") pod \"8010ebda-31a8-4732-88e0-758c754d2cda\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " Oct 30 19:04:45.479266 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.479171 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\" (UniqueName: \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\") pod \"8010ebda-31a8-4732-88e0-758c754d2cda\" (UID: \"8010ebda-31a8-4732-88e0-758c754d2cda\") " Oct 30 19:04:45.481394 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.481363 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs" pod "8010ebda-31a8-4732-88e0-758c754d2cda" (UID: "8010ebda-31a8-4732-88e0-758c754d2cda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:04:45.481482 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.481418 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm" pod "8010ebda-31a8-4732-88e0-758c754d2cda" (UID: "8010ebda-31a8-4732-88e0-758c754d2cda"). InnerVolumeSpecName "kube-api-access-n86pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:04:45.580654 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.580620 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8010ebda-31a8-4732-88e0-758c754d2cda-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:04:45.580797 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:45.580674 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n86pm\" (UniqueName: \"kubernetes.io/projected/8010ebda-31a8-4732-88e0-758c754d2cda-kube-api-access-n86pm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:04:46.247145 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:46.247079 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" event={"ID":"8010ebda-31a8-4732-88e0-758c754d2cda","Type":"ContainerDied","Data":"7904e49607ad02478cab723be7cf6c9140e5717ebb8941293ff4546f1dc9de7c"} Oct 30 19:04:46.247145 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:46.247131 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh" Oct 30 19:04:46.247656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:46.247180 5869 scope.go:122] "RemoveContainer" containerID="9f2e77664fdb11dc3eef3575bbd976f7c2af07717be8b56a1524b10951fb5f3b" Oct 30 19:04:46.406904 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:46.406842 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:04:46.454173 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:46.454144 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-pzcrh"] Oct 30 19:04:48.041532 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:48.041499 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" path="/var/lib/kubelet/pods/8010ebda-31a8-4732-88e0-758c754d2cda/volumes" Oct 30 19:04:58.293360 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.293323 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:04:58.293749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.293735 5869 cpu_manager.go:414] "containerMap: removing container" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" Oct 30 19:04:58.293794 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.293750 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" Oct 30 19:04:58.293831 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.293824 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="8010ebda-31a8-4732-88e0-758c754d2cda" containerName="main" Oct 30 19:04:58.298375 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.298358 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.381596 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.381571 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:04:58.426987 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.426963 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\"" Oct 30 19:04:58.427150 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.427055 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-lzg96\"" Oct 30 19:04:58.427150 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.427055 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-zj5dc\"" Oct 30 19:04:58.427150 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.427147 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\"" Oct 30 19:04:58.427512 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.427499 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\"" Oct 30 19:04:58.489844 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.489819 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jlfw\" (UniqueName: \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.490019 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.490000 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.591231 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.591145 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.591231 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.591203 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jlfw\" (UniqueName: \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.593837 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.593815 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.648380 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.648349 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jlfw\" (UniqueName: \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\") pod \"auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:58.911406 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:58.911334 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:04:59.281031 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:04:59.280993 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:04:59.286379 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:59.286353 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:04:59.288467 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:04:59.288441 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81570df6_41b4_4acb_9a1e_5e4e0d3d9ee5.slice/crio-0406b43def017dc6c6a3f3739c449e8e910afffe3b946b190a03143b113495d4 WatchSource:0}: Error finding container 0406b43def017dc6c6a3f3739c449e8e910afffe3b946b190a03143b113495d4: Status 404 returned error can't find the container with id 0406b43def017dc6c6a3f3739c449e8e910afffe3b946b190a03143b113495d4 Oct 30 19:04:59.307987 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:04:59.307959 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" event={"ID":"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5","Type":"ContainerStarted","Data":"0406b43def017dc6c6a3f3739c449e8e910afffe3b946b190a03143b113495d4"} Oct 30 19:05:00.316006 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:00.315954 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" event={"ID":"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5","Type":"ContainerStarted","Data":"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3"} Oct 30 19:05:00.316602 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:00.316189 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:05:07.186386 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:07.186299 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:05:07.299186 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:07.299129 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podStartSLOduration=9.29908443 podStartE2EDuration="9.29908443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:00.445675417 +0000 UTC m=+1624.517499497" watchObservedRunningTime="2026-10-30 19:05:07.29908443 +0000 UTC m=+1631.370908505" Oct 30 19:05:18.074080 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:18.074049 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:05:27.186321 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:27.186271 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:05:29.283976 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:05:29.283935 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:05:31.322515 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:31.322478 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:05:33.094081 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:33.094013 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" containerID="cri-o://3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1" gracePeriod=45 Oct 30 19:05:37.187201 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:37.187157 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:05:47.186246 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:47.186199 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:05:47.186679 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:47.186307 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:05:57.186771 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:05:57.186726 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:05:59.281675 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:05:59.281636 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:06:03.351658 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.351635 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:06:03.437993 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.437968 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\" (UniqueName: \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\") pod \"78bdd79e-0d65-4879-9434-4a76087d2bc0\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " Oct 30 19:06:03.438150 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.438016 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\" (UniqueName: \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\") pod \"78bdd79e-0d65-4879-9434-4a76087d2bc0\" (UID: \"78bdd79e-0d65-4879-9434-4a76087d2bc0\") " Oct 30 19:06:03.440230 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.440206 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs" pod "78bdd79e-0d65-4879-9434-4a76087d2bc0" (UID: "78bdd79e-0d65-4879-9434-4a76087d2bc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:06:03.440333 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.440314 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67" pod "78bdd79e-0d65-4879-9434-4a76087d2bc0" (UID: "78bdd79e-0d65-4879-9434-4a76087d2bc0"). InnerVolumeSpecName "kube-api-access-tlc67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:06:03.539204 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.539165 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bdd79e-0d65-4879-9434-4a76087d2bc0-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:06:03.539204 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.539206 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlc67\" (UniqueName: \"kubernetes.io/projected/78bdd79e-0d65-4879-9434-4a76087d2bc0-kube-api-access-tlc67\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:06:03.622774 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.622748 5869 generic.go:356] "Generic (PLEG): container finished" podID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerID="3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1" exitCode=0 Oct 30 19:06:03.622877 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.622795 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" event={"ID":"78bdd79e-0d65-4879-9434-4a76087d2bc0","Type":"ContainerDied","Data":"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1"} Oct 30 19:06:03.622877 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.622829 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" event={"ID":"78bdd79e-0d65-4879-9434-4a76087d2bc0","Type":"ContainerDied","Data":"d72e2a7fdf9f9f8027d50444d3a63badbdc82d4934e1d05e0d0a29759caadaa2"} Oct 30 19:06:03.622877 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.622850 5869 scope.go:122] "RemoveContainer" containerID="3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1" Oct 30 19:06:03.622877 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.622795 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp" Oct 30 19:06:03.631637 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.631618 5869 scope.go:122] "RemoveContainer" containerID="3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1" Oct 30 19:06:03.631884 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:06:03.631866 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1\": container with ID starting with 3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1 not found: ID does not exist" containerID="3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1" Oct 30 19:06:03.631948 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.631891 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1"} err="failed to get container status \"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1\": rpc error: code = NotFound desc = could not find container \"3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1\": container with ID starting with 3235b463ff370d4c38e742b0219cce2fdcd5d113cf5880014f57c2a17fc99ae1 not found: ID does not exist" Oct 30 19:06:03.781289 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.781235 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:06:03.786397 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:03.786378 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666ggppp"] Oct 30 19:06:04.040973 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:04.040911 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" path="/var/lib/kubelet/pods/78bdd79e-0d65-4879-9434-4a76087d2bc0/volumes" Oct 30 19:06:15.207723 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.207683 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:06:15.208315 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.208295 5869 cpu_manager.go:414] "containerMap: removing container" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" Oct 30 19:06:15.208397 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.208318 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" Oct 30 19:06:15.208451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.208411 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="78bdd79e-0d65-4879-9434-4a76087d2bc0" containerName="main" Oct 30 19:06:15.214135 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.213408 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.266539 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.266504 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:06:15.281450 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.281415 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\"" Oct 30 19:06:15.281586 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.281515 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\"" Oct 30 19:06:15.312223 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.312201 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-svgft\"" Oct 30 19:06:15.312223 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.312217 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-5mrjf\"" Oct 30 19:06:15.312405 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.312330 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\"" Oct 30 19:06:15.336384 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.336357 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2d7g\" (UniqueName: \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.336504 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.336409 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.437141 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.437081 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2d7g\" (UniqueName: \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.437273 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.437169 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.439736 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.439714 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.494234 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.494176 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2d7g\" (UniqueName: \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.528781 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.528758 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:15.896484 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:15.896463 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:06:15.898743 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:06:15.898716 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d0a1cd_7bf5_4080_addd_39a950197ad0.slice/crio-5b1ea8cac983b728f3bc51f81ae23be98bb586b2590b86f058fdf5ddb4001570 WatchSource:0}: Error finding container 5b1ea8cac983b728f3bc51f81ae23be98bb586b2590b86f058fdf5ddb4001570: Status 404 returned error can't find the container with id 5b1ea8cac983b728f3bc51f81ae23be98bb586b2590b86f058fdf5ddb4001570 Oct 30 19:06:16.683636 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:16.683603 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" event={"ID":"b5d0a1cd-7bf5-4080-addd-39a950197ad0","Type":"ContainerStarted","Data":"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263"} Oct 30 19:06:16.684011 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:16.683660 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" event={"ID":"b5d0a1cd-7bf5-4080-addd-39a950197ad0","Type":"ContainerStarted","Data":"5b1ea8cac983b728f3bc51f81ae23be98bb586b2590b86f058fdf5ddb4001570"} Oct 30 19:06:16.684011 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:16.683716 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:16.799135 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:16.799074 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podStartSLOduration=1.7990622040000002 podStartE2EDuration="1.799062204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:16.796896333 +0000 UTC m=+1700.868720422" watchObservedRunningTime="2026-10-30 19:06:16.799062204 +0000 UTC m=+1700.870886278" Oct 30 19:06:29.281112 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:06:29.281060 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:06:47.690295 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:47.690264 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:06:58.147251 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:06:58.147215 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:06:59.280613 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:06:59.280571 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:07:07.690293 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:07.690245 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:13.167678 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:13.167619 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" containerID="cri-o://2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263" gracePeriod=45 Oct 30 19:07:17.689864 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:17.689820 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:25.260911 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:25.260877 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:07:27.689614 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:27.689572 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:27.690107 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:27.689695 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:07:29.281264 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:07:29.281222 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:07:31.322993 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:31.322956 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:37.689960 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:37.689917 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:40.280827 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:40.280786 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" containerID="cri-o://a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3" gracePeriod=45 Oct 30 19:07:41.322450 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:41.322406 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:43.414137 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.414109 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:07:43.554997 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.554966 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\") pod \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " Oct 30 19:07:43.554997 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.555002 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\" (UniqueName: \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\") pod \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\" (UID: \"b5d0a1cd-7bf5-4080-addd-39a950197ad0\") " Oct 30 19:07:43.557350 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.557322 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs" pod "b5d0a1cd-7bf5-4080-addd-39a950197ad0" (UID: "b5d0a1cd-7bf5-4080-addd-39a950197ad0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:07:43.557449 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.557389 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g" pod "b5d0a1cd-7bf5-4080-addd-39a950197ad0" (UID: "b5d0a1cd-7bf5-4080-addd-39a950197ad0"). InnerVolumeSpecName "kube-api-access-r2d7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:43.655845 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.655820 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5d0a1cd-7bf5-4080-addd-39a950197ad0-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:43.655845 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:43.655844 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2d7g\" (UniqueName: \"kubernetes.io/projected/b5d0a1cd-7bf5-4080-addd-39a950197ad0-kube-api-access-r2d7g\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:44.080142 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.080084 5869 generic.go:356] "Generic (PLEG): container finished" podID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerID="2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263" exitCode=0 Oct 30 19:07:44.080270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.080151 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" Oct 30 19:07:44.080270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.080190 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" event={"ID":"b5d0a1cd-7bf5-4080-addd-39a950197ad0","Type":"ContainerDied","Data":"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263"} Oct 30 19:07:44.080270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.080244 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf" event={"ID":"b5d0a1cd-7bf5-4080-addd-39a950197ad0","Type":"ContainerDied","Data":"5b1ea8cac983b728f3bc51f81ae23be98bb586b2590b86f058fdf5ddb4001570"} Oct 30 19:07:44.080270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.080268 5869 scope.go:122] "RemoveContainer" containerID="2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263" Oct 30 19:07:44.088990 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.088963 5869 scope.go:122] "RemoveContainer" containerID="2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263" Oct 30 19:07:44.089255 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:07:44.089237 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263\": container with ID starting with 2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263 not found: ID does not exist" containerID="2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263" Oct 30 19:07:44.089322 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.089263 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263"} err="failed to get container status \"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263\": rpc error: code = NotFound desc = could not find container \"2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263\": container with ID starting with 2e54b54b924f40400a78f4094031f493d6f05de072a7328d2a5157b859ade263 not found: ID does not exist" Oct 30 19:07:44.239711 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.239688 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:07:44.286757 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:44.286737 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb6kzhtf"] Oct 30 19:07:46.040499 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:46.040458 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" path="/var/lib/kubelet/pods/b5d0a1cd-7bf5-4080-addd-39a950197ad0/volumes" Oct 30 19:07:51.322390 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:51.322345 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:07:51.322822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:51.322461 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:07:56.290565 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:07:56.290534 5869 scope.go:122] "RemoveContainer" containerID="e733287be49e275a993de31a0f008c43133be32224584400be7a1a2385c668fb" Oct 30 19:07:59.280139 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:07:59.280087 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:08:01.322816 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:01.322778 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:08:10.527654 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.527627 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:08:10.570732 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.570673 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\" (UniqueName: \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\") pod \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " Oct 30 19:08:10.570732 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.570717 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\" (UniqueName: \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\") pod \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\" (UID: \"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5\") " Oct 30 19:08:10.572898 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.572870 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs" pod "81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" (UID: "81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:08:10.572987 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.572929 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw" pod "81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" (UID: "81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5"). InnerVolumeSpecName "kube-api-access-8jlfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:08:10.671902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.671874 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:08:10.671902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:10.671899 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jlfw\" (UniqueName: \"kubernetes.io/projected/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5-kube-api-access-8jlfw\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:08:11.199281 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.199245 5869 generic.go:356] "Generic (PLEG): container finished" podID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerID="a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3" exitCode=0 Oct 30 19:08:11.199562 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.199299 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" event={"ID":"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5","Type":"ContainerDied","Data":"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3"} Oct 30 19:08:11.199562 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.199304 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" Oct 30 19:08:11.199562 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.199339 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf" event={"ID":"81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5","Type":"ContainerDied","Data":"0406b43def017dc6c6a3f3739c449e8e910afffe3b946b190a03143b113495d4"} Oct 30 19:08:11.199562 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.199362 5869 scope.go:122] "RemoveContainer" containerID="a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3" Oct 30 19:08:11.208371 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.208353 5869 scope.go:122] "RemoveContainer" containerID="a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3" Oct 30 19:08:11.208672 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:08:11.208652 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3\": container with ID starting with a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3 not found: ID does not exist" containerID="a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3" Oct 30 19:08:11.208733 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.208691 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3"} err="failed to get container status \"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3\": rpc error: code = NotFound desc = could not find container \"a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3\": container with ID starting with a035db0dc2fe086af5ce955c4e2473a749385c8ef2cc6b3d2347c808b9e47bd3 not found: ID does not exist" Oct 30 19:08:11.358995 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.358958 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:08:11.368743 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:11.368711 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-77676c6894prtjf"] Oct 30 19:08:12.041807 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:12.041765 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" path="/var/lib/kubelet/pods/81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5/volumes" Oct 30 19:08:19.338260 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338222 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:08:19.338658 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338602 5869 cpu_manager.go:414] "containerMap: removing container" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" Oct 30 19:08:19.338658 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338613 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" Oct 30 19:08:19.338658 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338628 5869 cpu_manager.go:414] "containerMap: removing container" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" Oct 30 19:08:19.338658 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338633 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" Oct 30 19:08:19.338786 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338702 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5d0a1cd-7bf5-4080-addd-39a950197ad0" containerName="main" Oct 30 19:08:19.338786 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.338711 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="81570df6-41b4-4acb-9a1e-5e4e0d3d9ee5" containerName="main" Oct 30 19:08:19.341615 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.341596 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.396461 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.396435 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:08:19.438926 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.438864 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-rz27j\"" Oct 30 19:08:19.438926 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.438910 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-5rzb2\"" Oct 30 19:08:19.439077 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.438920 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\"" Oct 30 19:08:19.439077 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.438987 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\"" Oct 30 19:08:19.440344 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.440328 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.440406 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.440388 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khzwg\" (UniqueName: \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.442464 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.442451 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\"" Oct 30 19:08:19.540903 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.540866 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.541056 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.540958 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khzwg\" (UniqueName: \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.543496 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.543474 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.596656 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.596602 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khzwg\" (UniqueName: \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\") pod \"auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:19.653726 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:19.653700 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:20.023143 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:20.023119 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:08:20.025655 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:08:20.025619 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d7ff49_8612_4cbf_9014_83fb47fca731.slice/crio-3e61fe6f91342b04ef31d9a330cb9eabd10cc58d63d5ad6d07c729b43fdb87b4 WatchSource:0}: Error finding container 3e61fe6f91342b04ef31d9a330cb9eabd10cc58d63d5ad6d07c729b43fdb87b4: Status 404 returned error can't find the container with id 3e61fe6f91342b04ef31d9a330cb9eabd10cc58d63d5ad6d07c729b43fdb87b4 Oct 30 19:08:20.242361 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:20.242324 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" event={"ID":"20d7ff49-8612-4cbf-9014-83fb47fca731","Type":"ContainerStarted","Data":"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099"} Oct 30 19:08:20.242509 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:20.242387 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" event={"ID":"20d7ff49-8612-4cbf-9014-83fb47fca731","Type":"ContainerStarted","Data":"3e61fe6f91342b04ef31d9a330cb9eabd10cc58d63d5ad6d07c729b43fdb87b4"} Oct 30 19:08:20.242509 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:20.242410 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:20.361371 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:20.361215 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podStartSLOduration=1.3612010190000001 podStartE2EDuration="1.361201019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:20.360406024 +0000 UTC m=+1824.432230101" watchObservedRunningTime="2026-10-30 19:08:20.361201019 +0000 UTC m=+1824.433025097" Oct 30 19:08:29.280730 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:08:29.280695 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:08:51.248942 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:08:51.248902 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:08:59.281360 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:08:59.281316 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:09:29.280233 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:09:29.280190 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:09:59.280888 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:09:59.280852 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:10:18.895934 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:18.895901 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:10:21.249490 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:21.249445 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:10:29.281105 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:10:29.281040 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:10:31.248801 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:31.248755 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:10:33.921383 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:33.921334 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" containerID="cri-o://be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099" gracePeriod=45 Oct 30 19:10:41.249346 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:41.249308 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:10:41.249739 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:41.249417 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:10:51.249203 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:10:51.249168 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:10:59.281982 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:10:59.281927 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:11:01.249648 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:01.249604 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:11:04.174422 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.174399 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:11:04.219730 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.219677 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\" (UniqueName: \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\") pod \"20d7ff49-8612-4cbf-9014-83fb47fca731\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " Oct 30 19:11:04.219730 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.219708 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\" (UniqueName: \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\") pod \"20d7ff49-8612-4cbf-9014-83fb47fca731\" (UID: \"20d7ff49-8612-4cbf-9014-83fb47fca731\") " Oct 30 19:11:04.221892 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.221867 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs" pod "20d7ff49-8612-4cbf-9014-83fb47fca731" (UID: "20d7ff49-8612-4cbf-9014-83fb47fca731"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:11:04.221982 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.221910 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg" pod "20d7ff49-8612-4cbf-9014-83fb47fca731" (UID: "20d7ff49-8612-4cbf-9014-83fb47fca731"). InnerVolumeSpecName "kube-api-access-khzwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:11:04.320449 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.320428 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d7ff49-8612-4cbf-9014-83fb47fca731-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:11:04.320550 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:04.320451 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khzwg\" (UniqueName: \"kubernetes.io/projected/20d7ff49-8612-4cbf-9014-83fb47fca731-kube-api-access-khzwg\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:11:05.007074 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.007041 5869 generic.go:356] "Generic (PLEG): container finished" podID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerID="be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099" exitCode=0 Oct 30 19:11:05.007270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.007115 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" Oct 30 19:11:05.007270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.007141 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" event={"ID":"20d7ff49-8612-4cbf-9014-83fb47fca731","Type":"ContainerDied","Data":"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099"} Oct 30 19:11:05.007270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.007202 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh" event={"ID":"20d7ff49-8612-4cbf-9014-83fb47fca731","Type":"ContainerDied","Data":"3e61fe6f91342b04ef31d9a330cb9eabd10cc58d63d5ad6d07c729b43fdb87b4"} Oct 30 19:11:05.007270 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.007226 5869 scope.go:122] "RemoveContainer" containerID="be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099" Oct 30 19:11:05.016908 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.016891 5869 scope.go:122] "RemoveContainer" containerID="be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099" Oct 30 19:11:05.017261 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:11:05.017242 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099\": container with ID starting with be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099 not found: ID does not exist" containerID="be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099" Oct 30 19:11:05.017318 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.017270 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099"} err="failed to get container status \"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099\": rpc error: code = NotFound desc = could not find container \"be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099\": container with ID starting with be098463edcb158fe4d3e08fa42b47656694937b05735f1f117c68b7165ed099 not found: ID does not exist" Oct 30 19:11:05.165960 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.165896 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:11:05.212963 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:05.212937 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-5d987cbbfd-s9hsh"] Oct 30 19:11:06.040295 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:11:06.040264 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" path="/var/lib/kubelet/pods/20d7ff49-8612-4cbf-9014-83fb47fca731/volumes" Oct 30 19:11:29.279549 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:11:29.279514 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:11:59.281465 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:11:59.281422 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:12:17.553828 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.553793 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:12:17.554312 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.554205 5869 cpu_manager.go:414] "containerMap: removing container" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" Oct 30 19:12:17.554312 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.554216 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" Oct 30 19:12:17.554312 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.554287 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="20d7ff49-8612-4cbf-9014-83fb47fca731" containerName="main" Oct 30 19:12:17.557455 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.557434 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.602047 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.602015 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjlbm\" (UniqueName: \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.602267 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.602088 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.617640 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.617611 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:12:17.657935 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.657895 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\"" Oct 30 19:12:17.658078 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.657938 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-z7tcw\"" Oct 30 19:12:17.658078 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.657951 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-lvtpg\"" Oct 30 19:12:17.658078 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.657951 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\"" Oct 30 19:12:17.663713 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.663695 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\"" Oct 30 19:12:17.703050 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.703014 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjlbm\" (UniqueName: \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.703223 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.703136 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.706363 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.706340 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.761266 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.761248 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjlbm\" (UniqueName: \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:17.869902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:17.869844 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:18.234643 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:18.234609 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:12:18.237255 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:12:18.237227 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6af526_5a49_47b6_b888_ead45a8e04bf.slice/crio-ba6c3f9b1b5d648ad85ebc0813489e7971da449b75f28c0e6f227db9cf2a3fb2 WatchSource:0}: Error finding container ba6c3f9b1b5d648ad85ebc0813489e7971da449b75f28c0e6f227db9cf2a3fb2: Status 404 returned error can't find the container with id ba6c3f9b1b5d648ad85ebc0813489e7971da449b75f28c0e6f227db9cf2a3fb2 Oct 30 19:12:18.330065 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:18.330038 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" event={"ID":"fc6af526-5a49-47b6-b888-ead45a8e04bf","Type":"ContainerStarted","Data":"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f"} Oct 30 19:12:18.330204 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:18.330085 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" event={"ID":"fc6af526-5a49-47b6-b888-ead45a8e04bf","Type":"ContainerStarted","Data":"ba6c3f9b1b5d648ad85ebc0813489e7971da449b75f28c0e6f227db9cf2a3fb2"} Oct 30 19:12:18.330257 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:18.330203 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:18.442479 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:18.442427 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podStartSLOduration=1.4424147170000001 podStartE2EDuration="1.442414717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:18.441882255 +0000 UTC m=+2062.513706332" watchObservedRunningTime="2026-10-30 19:12:18.442414717 +0000 UTC m=+2062.514238829" Oct 30 19:12:29.280737 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:12:29.280684 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:12:49.337067 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:12:49.337037 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:12:59.282757 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:12:59.282714 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:13:29.279302 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:13:29.279262 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:13:59.279027 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:13:59.278983 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:14:15.510639 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:15.510558 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:14:19.336975 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:19.336930 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:14:29.279045 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:14:29.279004 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:14:29.336542 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:29.336506 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:14:30.530804 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:30.530742 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" containerID="cri-o://9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f" gracePeriod=45 Oct 30 19:14:39.336888 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:39.336852 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:14:39.337304 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:39.336971 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:14:49.336061 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:49.336024 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:14:59.281943 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:14:59.281903 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:14:59.336473 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:14:59.336443 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:15:00.771467 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.771445 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:15:00.786019 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.785994 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\" (UniqueName: \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\") pod \"fc6af526-5a49-47b6-b888-ead45a8e04bf\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " Oct 30 19:15:00.786112 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.786049 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\") pod \"fc6af526-5a49-47b6-b888-ead45a8e04bf\" (UID: \"fc6af526-5a49-47b6-b888-ead45a8e04bf\") " Oct 30 19:15:00.788207 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.788155 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs" pod "fc6af526-5a49-47b6-b888-ead45a8e04bf" (UID: "fc6af526-5a49-47b6-b888-ead45a8e04bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:15:00.788295 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.788212 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm" pod "fc6af526-5a49-47b6-b888-ead45a8e04bf" (UID: "fc6af526-5a49-47b6-b888-ead45a8e04bf"). InnerVolumeSpecName "kube-api-access-gjlbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:00.887450 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.887423 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjlbm\" (UniqueName: \"kubernetes.io/projected/fc6af526-5a49-47b6-b888-ead45a8e04bf-kube-api-access-gjlbm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:00.887450 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:00.887447 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc6af526-5a49-47b6-b888-ead45a8e04bf-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:01.045679 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.045601 5869 generic.go:356] "Generic (PLEG): container finished" podID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerID="9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f" exitCode=0 Oct 30 19:15:01.045679 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.045657 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" Oct 30 19:15:01.045825 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.045689 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" event={"ID":"fc6af526-5a49-47b6-b888-ead45a8e04bf","Type":"ContainerDied","Data":"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f"} Oct 30 19:15:01.045825 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.045765 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg" event={"ID":"fc6af526-5a49-47b6-b888-ead45a8e04bf","Type":"ContainerDied","Data":"ba6c3f9b1b5d648ad85ebc0813489e7971da449b75f28c0e6f227db9cf2a3fb2"} Oct 30 19:15:01.045825 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.045787 5869 scope.go:122] "RemoveContainer" containerID="9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f" Oct 30 19:15:01.055234 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.055214 5869 scope.go:122] "RemoveContainer" containerID="9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f" Oct 30 19:15:01.055496 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:15:01.055472 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f\": container with ID starting with 9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f not found: ID does not exist" containerID="9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f" Oct 30 19:15:01.055557 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.055499 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f"} err="failed to get container status \"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f\": rpc error: code = NotFound desc = could not find container \"9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f\": container with ID starting with 9335f8587d85b946077fca303921982e639fe02a72b0fd0ba37a8782c08fbe4f not found: ID does not exist" Oct 30 19:15:01.203811 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.203788 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:15:01.209526 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:01.209504 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schex5crg"] Oct 30 19:15:02.041224 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:02.041186 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" path="/var/lib/kubelet/pods/fc6af526-5a49-47b6-b888-ead45a8e04bf/volumes" Oct 30 19:15:12.815104 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.815055 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:15:12.815474 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.815424 5869 cpu_manager.go:414] "containerMap: removing container" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" Oct 30 19:15:12.815474 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.815435 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" Oct 30 19:15:12.815548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.815504 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc6af526-5a49-47b6-b888-ead45a8e04bf" containerName="main" Oct 30 19:15:12.819891 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.819870 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:12.872158 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.872132 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:15:12.885153 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.885127 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:12.885283 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.885167 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8vf\" (UniqueName: \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:12.918135 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.918110 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"kube-root-ca.crt\"" Oct 30 19:15:12.918284 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.918260 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"openshift-service-ca.crt\"" Oct 30 19:15:12.918404 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.918305 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisvc-model-fb-opt-125m-oci-r-0fba19de-epp-sa-dockercfg-5g5zj\"" Oct 30 19:15:12.918404 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.918364 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisv20d1e8ef181cac7effb0fab7b8590b15-kserve-self-signed-certs\"" Oct 30 19:15:12.918498 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.918434 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"default-dockercfg-2278p\"" Oct 30 19:15:12.986624 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.986589 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:12.986793 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.986640 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8vf\" (UniqueName: \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:12.989206 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:12.989153 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:13.042919 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:13.042884 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8vf\" (UniqueName: \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:13.132753 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:13.132698 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:13.494045 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:13.494017 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:15:13.496828 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:15:13.496800 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c67ffc4_0ebf_4723_a180_1df7f8c6ee07.slice/crio-e2718d98e25eba49f9dfdfaf241ecb86717391230d5240ffdba47be41370cb8c WatchSource:0}: Error finding container e2718d98e25eba49f9dfdfaf241ecb86717391230d5240ffdba47be41370cb8c: Status 404 returned error can't find the container with id e2718d98e25eba49f9dfdfaf241ecb86717391230d5240ffdba47be41370cb8c Oct 30 19:15:14.100078 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:14.100037 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" event={"ID":"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07","Type":"ContainerStarted","Data":"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86"} Oct 30 19:15:14.100477 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:14.100115 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" event={"ID":"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07","Type":"ContainerStarted","Data":"e2718d98e25eba49f9dfdfaf241ecb86717391230d5240ffdba47be41370cb8c"} Oct 30 19:15:14.100477 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:14.100273 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:14.210681 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:14.210634 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podStartSLOduration=2.2106221 podStartE2EDuration="2.2106221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:14.209731747 +0000 UTC m=+2238.281555823" watchObservedRunningTime="2026-10-30 19:15:14.2106221 +0000 UTC m=+2238.282446211" Oct 30 19:15:29.279441 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:15:29.279404 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:15:44.077644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:44.077612 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 19:15:44.078070 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:44.077831 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" containerID="cri-o://2b59ab624ba9f0e5de6e624bb8ad2853656b9f7a53423ec522c8c464dcb2e645" gracePeriod=30 Oct 30 19:15:45.106034 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:45.106002 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:15:49.256009 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.255979 5869 generic.go:356] "Generic (PLEG): container finished" podID="21472f59-810b-4571-8e3b-eb160c4ff080" containerID="2b59ab624ba9f0e5de6e624bb8ad2853656b9f7a53423ec522c8c464dcb2e645" exitCode=0 Oct 30 19:15:49.256452 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.256056 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" event={"ID":"21472f59-810b-4571-8e3b-eb160c4ff080","Type":"ContainerDied","Data":"2b59ab624ba9f0e5de6e624bb8ad2853656b9f7a53423ec522c8c464dcb2e645"} Oct 30 19:15:49.331757 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.331734 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 19:15:49.495806 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.495777 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.495952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.495826 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.495952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.495868 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.495952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.495906 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.496205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.495964 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.496205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.496004 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.496205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.496041 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.496205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.496077 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.496205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.496169 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\") pod \"21472f59-810b-4571-8e3b-eb160c4ff080\" (UID: \"21472f59-810b-4571-8e3b-eb160c4ff080\") " Oct 30 19:15:49.497188 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.497162 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:49.497413 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.497386 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:49.497603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.497582 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:49.498181 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.497772 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:49.498181 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.498144 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:49.499046 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.499021 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:15:49.500023 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.499996 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "kube-api-access-sbkm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:49.500147 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.500035 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:49.500697 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.500668 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token" pod "21472f59-810b-4571-8e3b-eb160c4ff080" (UID: "21472f59-810b-4571-8e3b-eb160c4ff080"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:49.597329 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597306 5869 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-credential-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597329 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597331 5869 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-envoy\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597344 5869 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21472f59-810b-4571-8e3b-eb160c4ff080-istiod-ca-cert\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597352 5869 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-istio-token\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597360 5869 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-istio-data\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597371 5869 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597379 5869 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21472f59-810b-4571-8e3b-eb160c4ff080-istio-podinfo\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597388 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbkm9\" (UniqueName: \"kubernetes.io/projected/21472f59-810b-4571-8e3b-eb160c4ff080-kube-api-access-sbkm9\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:49.597465 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:49.597397 5869 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21472f59-810b-4571-8e3b-eb160c4ff080-workload-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:15:50.261606 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:50.261580 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" Oct 30 19:15:50.261606 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:50.261594 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l" event={"ID":"21472f59-810b-4571-8e3b-eb160c4ff080","Type":"ContainerDied","Data":"ad2d54fe442196a2d988fdc8218c278b2b5388bf430482075b305935583c7fa2"} Oct 30 19:15:50.262050 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:50.261647 5869 scope.go:122] "RemoveContainer" containerID="2b59ab624ba9f0e5de6e624bb8ad2853656b9f7a53423ec522c8c464dcb2e645" Oct 30 19:15:50.497324 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:50.497290 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 19:15:50.520790 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:50.520758 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-lb64l"] Oct 30 19:15:52.040998 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:15:52.040959 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" path="/var/lib/kubelet/pods/21472f59-810b-4571-8e3b-eb160c4ff080/volumes" Oct 30 19:15:59.280264 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:15:59.280226 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:16:29.280959 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:16:29.280916 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:16:59.281644 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:16:59.281603 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:17:29.280533 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:17:29.280438 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:17:30.403703 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:30.403671 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:17:35.106228 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:35.106172 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:17:45.105796 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:45.105759 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:17:45.423731 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:45.423652 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" containerID="cri-o://bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86" gracePeriod=45 Oct 30 19:17:55.105628 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:55.105594 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:17:55.106118 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:17:55.105692 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:17:59.282610 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:17:59.282574 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:18:05.106070 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:05.106026 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:18:15.106210 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.106173 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:18:15.674565 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.674542 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:18:15.759535 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.759503 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\") pod \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " Oct 30 19:18:15.759649 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.759628 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\" (UniqueName: \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\") pod \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\" (UID: \"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07\") " Oct 30 19:18:15.761853 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.761827 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf" pod "2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" (UID: "2c67ffc4-0ebf-4723-a180-1df7f8c6ee07"). InnerVolumeSpecName "kube-api-access-cb8vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:15.761949 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.761851 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs" pod "2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" (UID: "2c67ffc4-0ebf-4723-a180-1df7f8c6ee07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:15.860831 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.860803 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb8vf\" (UniqueName: \"kubernetes.io/projected/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-kube-api-access-cb8vf\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:18:15.860831 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.860829 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:18:15.951714 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.951686 5869 generic.go:356] "Generic (PLEG): container finished" podID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerID="bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86" exitCode=0 Oct 30 19:18:15.951843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.951734 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" Oct 30 19:18:15.951843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.951749 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" event={"ID":"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07","Type":"ContainerDied","Data":"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86"} Oct 30 19:18:15.951843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.951789 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx" event={"ID":"2c67ffc4-0ebf-4723-a180-1df7f8c6ee07","Type":"ContainerDied","Data":"e2718d98e25eba49f9dfdfaf241ecb86717391230d5240ffdba47be41370cb8c"} Oct 30 19:18:15.951843 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.951811 5869 scope.go:122] "RemoveContainer" containerID="bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86" Oct 30 19:18:15.963437 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.963414 5869 scope.go:122] "RemoveContainer" containerID="bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86" Oct 30 19:18:15.963790 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:18:15.963768 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86\": container with ID starting with bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86 not found: ID does not exist" containerID="bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86" Oct 30 19:18:15.963864 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:15.963802 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86"} err="failed to get container status \"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86\": rpc error: code = NotFound desc = could not find container \"bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86\": container with ID starting with bf02fe899cf33f035f7fbb7702f409f7e3507fda2ba13f55ae6fbeff88942b86 not found: ID does not exist" Oct 30 19:18:16.111015 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:16.110959 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:18:16.118140 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:16.118116 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-sche5xstx"] Oct 30 19:18:18.040988 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:18.040951 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" path="/var/lib/kubelet/pods/2c67ffc4-0ebf-4723-a180-1df7f8c6ee07/volumes" Oct 30 19:18:29.281913 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:18:29.281871 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:18:49.894124 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.893721 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894448 5869 cpu_manager.go:414] "containerMap: removing container" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894480 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894512 5869 cpu_manager.go:414] "containerMap: removing container" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894520 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894653 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c67ffc4-0ebf-4723-a180-1df7f8c6ee07" containerName="main" Oct 30 19:18:49.902347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.894686 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="21472f59-810b-4571-8e3b-eb160c4ff080" containerName="istio-proxy" Oct 30 19:18:49.922387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.922359 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:49.949514 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:49.949489 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:18:50.042164 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042130 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.042333 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042222 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q69n\" (UniqueName: \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.042834 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042820 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-fj8mt\"" Oct 30 19:18:50.042895 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042883 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-k76hw\"" Oct 30 19:18:50.042933 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042920 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\"" Oct 30 19:18:50.042989 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.042979 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\"" Oct 30 19:18:50.043050 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.043040 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\"" Oct 30 19:18:50.142932 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.142900 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.143124 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.142949 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q69n\" (UniqueName: \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.146152 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.145871 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.217973 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.217941 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q69n\" (UniqueName: \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.238361 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.238330 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:50.711117 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:50.711029 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:18:50.722792 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:18:50.722767 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081f2627_c8c7_45be_880f_310e9b68b78d.slice/crio-87d5ee9c022cc94092e12e626619ed44676f8a19758c7153dc8f75c50a932663 WatchSource:0}: Error finding container 87d5ee9c022cc94092e12e626619ed44676f8a19758c7153dc8f75c50a932663: Status 404 returned error can't find the container with id 87d5ee9c022cc94092e12e626619ed44676f8a19758c7153dc8f75c50a932663 Oct 30 19:18:51.136271 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:51.136180 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" event={"ID":"081f2627-c8c7-45be-880f-310e9b68b78d","Type":"ContainerStarted","Data":"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12"} Oct 30 19:18:51.136271 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:51.136238 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" event={"ID":"081f2627-c8c7-45be-880f-310e9b68b78d","Type":"ContainerStarted","Data":"87d5ee9c022cc94092e12e626619ed44676f8a19758c7153dc8f75c50a932663"} Oct 30 19:18:51.136793 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:51.136331 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:18:51.251282 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:18:51.251233 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podStartSLOduration=2.251218474 podStartE2EDuration="2.251218474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:51.250347852 +0000 UTC m=+2455.322171943" watchObservedRunningTime="2026-10-30 19:18:51.251218474 +0000 UTC m=+2455.323042550" Oct 30 19:18:59.279159 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:18:59.279120 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:19:15.768839 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.768807 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:19:15.773410 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.773391 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966061 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966018 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966254 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966068 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966254 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966186 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966254 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966225 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966278 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966323 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966354 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlqqn\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966513 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966379 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.966513 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.966488 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:15.974506 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:15.974476 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:19:16.067543 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067477 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067543 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067510 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067543 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067533 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067751 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067553 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067751 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067674 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067751 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067744 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067913 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067783 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067913 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067829 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlqqn\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.067913 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067867 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.068077 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.067969 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.068168 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.068071 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.068227 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.068160 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.068227 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.068189 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.068683 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.068657 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.070112 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.070077 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.070377 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.070360 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.124394 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.124365 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.124578 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.124562 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlqqn\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-bctpm\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.385412 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.385346 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:16.800570 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.800512 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:19:16.803170 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:19:16.803137 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970d989d_e6c8_4fe9_bb41_d84a5afe0c17.slice/crio-85f90405c655cbac156cbc65903e10f83ebd156f1db5f47bee1a57d3aa170b50 WatchSource:0}: Error finding container 85f90405c655cbac156cbc65903e10f83ebd156f1db5f47bee1a57d3aa170b50: Status 404 returned error can't find the container with id 85f90405c655cbac156cbc65903e10f83ebd156f1db5f47bee1a57d3aa170b50 Oct 30 19:19:16.805219 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.805177 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:19:16.805317 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.805258 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:19:16.805317 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:16.805290 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:19:17.246258 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.246209 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" event={"ID":"970d989d-e6c8-4fe9-bb41-d84a5afe0c17","Type":"ContainerStarted","Data":"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836"} Oct 30 19:19:17.246447 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.246274 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" event={"ID":"970d989d-e6c8-4fe9-bb41-d84a5afe0c17","Type":"ContainerStarted","Data":"85f90405c655cbac156cbc65903e10f83ebd156f1db5f47bee1a57d3aa170b50"} Oct 30 19:19:17.391120 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.388798 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:17.393259 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.393218 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:17.393418 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.393301 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:17.408971 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:17.408911 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podStartSLOduration=2.408895476 podStartE2EDuration="2.408895476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:17.408768209 +0000 UTC m=+2481.480592286" watchObservedRunningTime="2026-10-30 19:19:17.408895476 +0000 UTC m=+2481.480719552" Oct 30 19:19:18.391118 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:18.390471 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:18.391118 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:18.390532 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:19.389534 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:19.389496 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:19.389841 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:19.389811 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:20.388131 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:20.387396 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:20.388131 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:20.387455 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:21.391119 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:21.387454 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:21.391119 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:21.387512 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:22.142852 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:22.142819 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:19:22.392118 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:22.391464 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-bctpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Oct 30 19:19:22.392118 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:22.391527 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Oct 30 19:19:23.389958 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:23.389931 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:23.390225 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:23.390205 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:23.390788 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:19:23.390774 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:19:29.279292 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:19:29.279255 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:19:59.279677 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:19:59.279640 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:20:29.279970 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:20:29.279889 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:20:33.970573 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:20:33.970540 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:20:42.142594 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:20:42.142558 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:20:48.990639 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:20:48.990578 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" containerID="cri-o://c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12" gracePeriod=45 Oct 30 19:20:52.143242 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:20:52.143207 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:20:59.282700 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:20:59.282659 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:21:02.142934 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:02.142900 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:21:02.143424 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:02.142996 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:21:12.142734 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:12.142694 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:21:19.251279 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.251218 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:21:19.274928 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.274906 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\" (UniqueName: \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\") pod \"081f2627-c8c7-45be-880f-310e9b68b78d\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " Oct 30 19:21:19.275061 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.275026 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\" (UniqueName: \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\") pod \"081f2627-c8c7-45be-880f-310e9b68b78d\" (UID: \"081f2627-c8c7-45be-880f-310e9b68b78d\") " Oct 30 19:21:19.277158 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.277132 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs" pod "081f2627-c8c7-45be-880f-310e9b68b78d" (UID: "081f2627-c8c7-45be-880f-310e9b68b78d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:21:19.277241 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.277179 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n" pod "081f2627-c8c7-45be-880f-310e9b68b78d" (UID: "081f2627-c8c7-45be-880f-310e9b68b78d"). InnerVolumeSpecName "kube-api-access-6q69n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:21:19.375584 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.375542 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081f2627-c8c7-45be-880f-310e9b68b78d-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:19.375584 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.375581 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q69n\" (UniqueName: \"kubernetes.io/projected/081f2627-c8c7-45be-880f-310e9b68b78d-kube-api-access-6q69n\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:21:19.813796 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.813757 5869 generic.go:356] "Generic (PLEG): container finished" podID="081f2627-c8c7-45be-880f-310e9b68b78d" containerID="c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12" exitCode=0 Oct 30 19:21:19.814148 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.814120 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" event={"ID":"081f2627-c8c7-45be-880f-310e9b68b78d","Type":"ContainerDied","Data":"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12"} Oct 30 19:21:19.814364 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.814344 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" event={"ID":"081f2627-c8c7-45be-880f-310e9b68b78d","Type":"ContainerDied","Data":"87d5ee9c022cc94092e12e626619ed44676f8a19758c7153dc8f75c50a932663"} Oct 30 19:21:19.814518 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.814502 5869 scope.go:122] "RemoveContainer" containerID="c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12" Oct 30 19:21:19.814767 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.814748 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9" Oct 30 19:21:19.834500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.833731 5869 scope.go:122] "RemoveContainer" containerID="c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12" Oct 30 19:21:19.834500 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:21:19.834014 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12\": container with ID starting with c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12 not found: ID does not exist" containerID="c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12" Oct 30 19:21:19.834500 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.834049 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12"} err="failed to get container status \"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12\": rpc error: code = NotFound desc = could not find container \"c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12\": container with ID starting with c2862f639695e9c0c791acc495d393873e0fcbfdf84120bb610f9e89f457ea12 not found: ID does not exist" Oct 30 19:21:19.977762 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:19.977730 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:21:20.024781 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:20.024751 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d548t6h9"] Oct 30 19:21:20.041790 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:20.041765 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" path="/var/lib/kubelet/pods/081f2627-c8c7-45be-880f-310e9b68b78d/volumes" Oct 30 19:21:29.280258 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:21:29.280223 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:21:31.080376 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.080340 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:21:31.080824 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.080711 5869 cpu_manager.go:414] "containerMap: removing container" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" Oct 30 19:21:31.080824 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.080722 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" Oct 30 19:21:31.080824 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.080787 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="081f2627-c8c7-45be-880f-310e9b68b78d" containerName="main" Oct 30 19:21:31.083822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.083803 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.181803 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.181773 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-g8jws\"" Oct 30 19:21:31.181952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.181800 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-epp-sa-dockercfg-8lhwd\"" Oct 30 19:21:31.181952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.181860 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\"" Oct 30 19:21:31.181952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.181879 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\"" Oct 30 19:21:31.183281 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.183251 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nklql\" (UniqueName: \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.183424 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.183406 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.184539 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.184522 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:21:31.235066 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.235037 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\"" Oct 30 19:21:31.284297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.284272 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.284402 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.284312 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nklql\" (UniqueName: \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.286797 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.286776 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.341152 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.341082 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nklql\" (UniqueName: \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.396273 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.396248 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.757852 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.757831 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:21:31.760395 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:21:31.760365 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74284a00_9b12_4452_b0bf_684315a6b200.slice/crio-5bcfb5f41e499a170c345b7c47d3dafa823371eea48dfb35576a8b76b1c7bb5d WatchSource:0}: Error finding container 5bcfb5f41e499a170c345b7c47d3dafa823371eea48dfb35576a8b76b1c7bb5d: Status 404 returned error can't find the container with id 5bcfb5f41e499a170c345b7c47d3dafa823371eea48dfb35576a8b76b1c7bb5d Oct 30 19:21:31.867346 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.867310 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" event={"ID":"74284a00-9b12-4452-b0bf-684315a6b200","Type":"ContainerStarted","Data":"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b"} Oct 30 19:21:31.867493 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.867367 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" event={"ID":"74284a00-9b12-4452-b0bf-684315a6b200","Type":"ContainerStarted","Data":"5bcfb5f41e499a170c345b7c47d3dafa823371eea48dfb35576a8b76b1c7bb5d"} Oct 30 19:21:31.867493 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.867396 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:21:31.978110 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:21:31.978054 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podStartSLOduration=0.978043104 podStartE2EDuration="978.043104ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:31.977667175 +0000 UTC m=+2616.049491254" watchObservedRunningTime="2026-10-30 19:21:31.978043104 +0000 UTC m=+2616.049867179" Oct 30 19:21:59.279490 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:21:59.279451 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:22:02.873798 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:22:02.873770 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:22:29.280793 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:22:29.280745 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:22:59.279829 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:22:59.279789 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:23:25.086753 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.086666 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:23:25.090113 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.090075 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.172504 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.172480 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\"" Oct 30 19:23:25.172504 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.172480 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\"" Oct 30 19:23:25.192752 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.192723 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-cbg8p\"" Oct 30 19:23:25.193803 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.193785 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-dqkpq\"" Oct 30 19:23:25.193912 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.193825 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\"" Oct 30 19:23:25.194974 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.194955 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:23:25.211297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.211278 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.211413 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.211325 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brgng\" (UniqueName: \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.312293 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.312258 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.312444 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.312322 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brgng\" (UniqueName: \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.314855 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.314828 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.373293 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.373240 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brgng\" (UniqueName: \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.402364 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.402337 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:25.787643 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:25.787619 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:23:25.790328 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:23:25.790301 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ed0729_22c5_46ee_a6e7_f1c8b43604bb.slice/crio-8594f21c82ac0c10e09567b630e2ab5d76072082d498396de27cf18d6b984957 WatchSource:0}: Error finding container 8594f21c82ac0c10e09567b630e2ab5d76072082d498396de27cf18d6b984957: Status 404 returned error can't find the container with id 8594f21c82ac0c10e09567b630e2ab5d76072082d498396de27cf18d6b984957 Oct 30 19:23:26.376253 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:26.376217 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" event={"ID":"37ed0729-22c5-46ee-a6e7-f1c8b43604bb","Type":"ContainerStarted","Data":"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e"} Oct 30 19:23:26.376631 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:26.376270 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" event={"ID":"37ed0729-22c5-46ee-a6e7-f1c8b43604bb","Type":"ContainerStarted","Data":"8594f21c82ac0c10e09567b630e2ab5d76072082d498396de27cf18d6b984957"} Oct 30 19:23:26.376631 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:26.376344 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:26.487850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:26.487804 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podStartSLOduration=1.48779174 podStartE2EDuration="1.48779174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:26.487141792 +0000 UTC m=+2730.558965873" watchObservedRunningTime="2026-10-30 19:23:26.48779174 +0000 UTC m=+2730.559615816" Oct 30 19:23:29.280566 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:23:29.280530 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:23:29.825254 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:29.825226 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:23:32.873921 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:32.873882 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:23:42.873277 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:42.873224 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:23:44.845444 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:44.845372 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" containerID="cri-o://c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b" gracePeriod=45 Oct 30 19:23:52.873822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:52.873781 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:23:52.876313 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:52.873913 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:23:57.382426 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:23:57.382394 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:23:59.282044 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:23:59.282006 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:24:02.873438 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:02.873395 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:24:12.873953 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:12.873907 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:24:15.097401 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.097374 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:24:15.111703 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.111656 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\" (UniqueName: \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\") pod \"74284a00-9b12-4452-b0bf-684315a6b200\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " Oct 30 19:24:15.111781 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.111743 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\" (UniqueName: \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\") pod \"74284a00-9b12-4452-b0bf-684315a6b200\" (UID: \"74284a00-9b12-4452-b0bf-684315a6b200\") " Oct 30 19:24:15.113955 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.113930 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql" pod "74284a00-9b12-4452-b0bf-684315a6b200" (UID: "74284a00-9b12-4452-b0bf-684315a6b200"). InnerVolumeSpecName "kube-api-access-nklql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:24:15.114052 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.114023 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs" pod "74284a00-9b12-4452-b0bf-684315a6b200" (UID: "74284a00-9b12-4452-b0bf-684315a6b200"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:24:15.212949 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.212901 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nklql\" (UniqueName: \"kubernetes.io/projected/74284a00-9b12-4452-b0bf-684315a6b200-kube-api-access-nklql\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:15.212949 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.212922 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74284a00-9b12-4452-b0bf-684315a6b200-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:15.603739 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.603707 5869 generic.go:356] "Generic (PLEG): container finished" podID="74284a00-9b12-4452-b0bf-684315a6b200" containerID="c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b" exitCode=0 Oct 30 19:24:15.603866 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.603758 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" Oct 30 19:24:15.603866 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.603794 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" event={"ID":"74284a00-9b12-4452-b0bf-684315a6b200","Type":"ContainerDied","Data":"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b"} Oct 30 19:24:15.603866 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.603851 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm" event={"ID":"74284a00-9b12-4452-b0bf-684315a6b200","Type":"ContainerDied","Data":"5bcfb5f41e499a170c345b7c47d3dafa823371eea48dfb35576a8b76b1c7bb5d"} Oct 30 19:24:15.603985 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.603872 5869 scope.go:122] "RemoveContainer" containerID="c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b" Oct 30 19:24:15.613349 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.613330 5869 scope.go:122] "RemoveContainer" containerID="c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b" Oct 30 19:24:15.613629 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:24:15.613598 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b\": container with ID starting with c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b not found: ID does not exist" containerID="c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b" Oct 30 19:24:15.613682 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.613636 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b"} err="failed to get container status \"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b\": rpc error: code = NotFound desc = could not find container \"c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b\": container with ID starting with c6a3fc8f41b77bcb96d0bb0ef808736c22f9a3befbb0f175f997dd914f49fe7b not found: ID does not exist" Oct 30 19:24:15.762440 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.762412 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:24:15.765192 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:15.765172 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-7lkvm"] Oct 30 19:24:16.041688 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:16.041658 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74284a00-9b12-4452-b0bf-684315a6b200" path="/var/lib/kubelet/pods/74284a00-9b12-4452-b0bf-684315a6b200/volumes" Oct 30 19:24:29.283931 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:24:29.283891 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:24:33.137061 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:33.137026 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:24:33.137538 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:33.137291 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" containerID="cri-o://3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836" gracePeriod=30 Oct 30 19:24:38.389134 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.389110 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:24:38.491030 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491005 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491166 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491041 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491251 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491231 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491309 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491273 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491309 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491296 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491412 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491322 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491412 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491356 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491571 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491535 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:24:38.491715 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491558 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:24:38.491715 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491620 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:24:38.491715 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491624 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491875 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491762 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\") pod \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\" (UID: \"970d989d-e6c8-4fe9-bb41-d84a5afe0c17\") " Oct 30 19:24:38.491875 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491778 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:24:38.491875 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491706 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:24:38.492007 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.491991 5869 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.492062 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.492014 5869 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-credential-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.492062 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.492029 5869 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istiod-ca-cert\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.492062 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.492042 5869 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-workload-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.492062 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.492055 5869 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-data\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.493647 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.493624 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:24:38.493868 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.493851 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:24:38.493963 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.493878 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "kube-api-access-mlqqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:24:38.494084 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.494066 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token" pod "970d989d-e6c8-4fe9-bb41-d84a5afe0c17" (UID: "970d989d-e6c8-4fe9-bb41-d84a5afe0c17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:24:38.592948 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.592923 5869 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-token\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.592948 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.592945 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlqqn\" (UniqueName: \"kubernetes.io/projected/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-kube-api-access-mlqqn\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.593082 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.592954 5869 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-envoy\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.593082 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.592963 5869 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/970d989d-e6c8-4fe9-bb41-d84a5afe0c17-istio-podinfo\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:24:38.710800 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.710770 5869 generic.go:356] "Generic (PLEG): container finished" podID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerID="3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836" exitCode=0 Oct 30 19:24:38.710988 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.710817 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" Oct 30 19:24:38.710988 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.710854 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" event={"ID":"970d989d-e6c8-4fe9-bb41-d84a5afe0c17","Type":"ContainerDied","Data":"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836"} Oct 30 19:24:38.710988 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.710916 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm" event={"ID":"970d989d-e6c8-4fe9-bb41-d84a5afe0c17","Type":"ContainerDied","Data":"85f90405c655cbac156cbc65903e10f83ebd156f1db5f47bee1a57d3aa170b50"} Oct 30 19:24:38.710988 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.710948 5869 scope.go:122] "RemoveContainer" containerID="3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836" Oct 30 19:24:38.720863 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.720848 5869 scope.go:122] "RemoveContainer" containerID="3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836" Oct 30 19:24:38.721159 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:24:38.721140 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836\": container with ID starting with 3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836 not found: ID does not exist" containerID="3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836" Oct 30 19:24:38.721247 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.721167 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836"} err="failed to get container status \"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836\": rpc error: code = NotFound desc = could not find container \"3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836\": container with ID starting with 3bf31ac3247af9707eeec0af98126b67dbd59b079f9d0e45ea819fe75d5d4836 not found: ID does not exist" Oct 30 19:24:38.876574 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.876513 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:24:38.881686 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:38.881664 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-bctpm"] Oct 30 19:24:40.041428 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:24:40.041393 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" path="/var/lib/kubelet/pods/970d989d-e6c8-4fe9-bb41-d84a5afe0c17/volumes" Oct 30 19:24:59.281969 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:24:59.281934 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:25:29.279964 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:25:29.279925 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:25:32.629812 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:32.629775 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:25:37.383272 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:37.383227 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:25:47.382884 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:47.382836 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:25:47.655445 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:47.655357 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" containerID="cri-o://76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e" gracePeriod=45 Oct 30 19:25:57.382815 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:57.382773 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:25:57.383345 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:25:57.382891 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:25:59.281076 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:25:59.281040 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:26:07.383084 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:07.382990 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:26:17.382526 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.382489 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:26:17.909151 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.909127 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:26:17.992159 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.992131 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\" (UniqueName: \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\") pod \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " Oct 30 19:26:17.992298 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.992191 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\" (UniqueName: \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\") pod \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\" (UID: \"37ed0729-22c5-46ee-a6e7-f1c8b43604bb\") " Oct 30 19:26:17.994391 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.994363 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng" pod "37ed0729-22c5-46ee-a6e7-f1c8b43604bb" (UID: "37ed0729-22c5-46ee-a6e7-f1c8b43604bb"). InnerVolumeSpecName "kube-api-access-brgng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:26:17.994501 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:17.994424 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs" pod "37ed0729-22c5-46ee-a6e7-f1c8b43604bb" (UID: "37ed0729-22c5-46ee-a6e7-f1c8b43604bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:26:18.093451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.093423 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:18.093451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.093447 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brgng\" (UniqueName: \"kubernetes.io/projected/37ed0729-22c5-46ee-a6e7-f1c8b43604bb-kube-api-access-brgng\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:18.158993 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.158961 5869 generic.go:356] "Generic (PLEG): container finished" podID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerID="76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e" exitCode=0 Oct 30 19:26:18.159130 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.159007 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" Oct 30 19:26:18.159130 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.159049 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" event={"ID":"37ed0729-22c5-46ee-a6e7-f1c8b43604bb","Type":"ContainerDied","Data":"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e"} Oct 30 19:26:18.159130 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.159122 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s" event={"ID":"37ed0729-22c5-46ee-a6e7-f1c8b43604bb","Type":"ContainerDied","Data":"8594f21c82ac0c10e09567b630e2ab5d76072082d498396de27cf18d6b984957"} Oct 30 19:26:18.159259 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.159151 5869 scope.go:122] "RemoveContainer" containerID="76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e" Oct 30 19:26:18.167681 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.167655 5869 scope.go:122] "RemoveContainer" containerID="76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e" Oct 30 19:26:18.167996 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:26:18.167977 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e\": container with ID starting with 76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e not found: ID does not exist" containerID="76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e" Oct 30 19:26:18.168056 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.168005 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e"} err="failed to get container status \"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e\": rpc error: code = NotFound desc = could not find container \"76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e\": container with ID starting with 76758b51fa48b1dee54c42172c6b72e427899a6137d3e8292d795fea2c3bb65e not found: ID does not exist" Oct 30 19:26:18.316915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.316861 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:26:18.322836 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:18.322815 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-x6v7s"] Oct 30 19:26:20.042278 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:20.042244 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" path="/var/lib/kubelet/pods/37ed0729-22c5-46ee-a6e7-f1c8b43604bb/volumes" Oct 30 19:26:29.281054 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:26:29.281016 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:26:49.225792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:49.225757 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 19:26:49.226387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:49.225988 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" containerID="cri-o://62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a" gracePeriod=30 Oct 30 19:26:54.471481 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.471457 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 19:26:54.592131 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592046 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592131 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592087 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592336 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592161 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592336 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592223 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592336 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592255 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592336 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592297 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592338 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592344 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:26:54.592548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592365 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592438 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\") pod \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\" (UID: \"59d9cb1f-4b80-4bc0-97e4-7e4464117328\") " Oct 30 19:26:54.592822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592550 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:26:54.592822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592565 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:26:54.592822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592674 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:26:54.592978 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592843 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:26:54.593042 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.592983 5869 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-data\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.593042 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.593004 5869 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.593042 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.593019 5869 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istiod-ca-cert\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.593042 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.593032 5869 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-workload-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.594528 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.594499 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "kube-api-access-8x9cm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:26:54.594744 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.594727 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:26:54.594855 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.594838 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:26:54.594964 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.594950 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token" pod "59d9cb1f-4b80-4bc0-97e4-7e4464117328" (UID: "59d9cb1f-4b80-4bc0-97e4-7e4464117328"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:26:54.694229 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.694204 5869 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-credential-socket\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.694229 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.694227 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x9cm\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-kube-api-access-8x9cm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.694370 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.694237 5869 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-token\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.694370 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.694245 5869 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-envoy\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:54.694370 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:54.694254 5869 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59d9cb1f-4b80-4bc0-97e4-7e4464117328-istio-podinfo\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:26:55.322468 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.322428 5869 generic.go:356] "Generic (PLEG): container finished" podID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerID="62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a" exitCode=0 Oct 30 19:26:55.322669 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.322484 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" Oct 30 19:26:55.322669 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.322526 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerDied","Data":"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a"} Oct 30 19:26:55.322669 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.322589 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf" event={"ID":"59d9cb1f-4b80-4bc0-97e4-7e4464117328","Type":"ContainerDied","Data":"217a4315087d7680da61a0d1c1ea1c45e957eb3be727b03f3cdf1c0209ae35cc"} Oct 30 19:26:55.322669 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.322655 5869 scope.go:122] "RemoveContainer" containerID="62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a" Oct 30 19:26:55.332414 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.332390 5869 scope.go:122] "RemoveContainer" containerID="e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643" Oct 30 19:26:55.340159 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.340133 5869 scope.go:122] "RemoveContainer" containerID="62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a" Oct 30 19:26:55.340435 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:26:55.340414 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a\": container with ID starting with 62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a not found: ID does not exist" containerID="62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a" Oct 30 19:26:55.340518 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.340447 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a"} err="failed to get container status \"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a\": rpc error: code = NotFound desc = could not find container \"62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a\": container with ID starting with 62b91a8bf6bf3b8d7ac300e9eaf227317b7bd27afed69a72f77a684c6820794a not found: ID does not exist" Oct 30 19:26:55.340518 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.340473 5869 scope.go:122] "RemoveContainer" containerID="e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643" Oct 30 19:26:55.340773 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:26:55.340754 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643\": container with ID starting with e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643 not found: ID does not exist" containerID="e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643" Oct 30 19:26:55.340856 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.340784 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643"} err="failed to get container status \"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643\": rpc error: code = NotFound desc = could not find container \"e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643\": container with ID starting with e6848b45fb84dfbb096f68e1cd422b52f861a1c44ec07818f80e541877d37643 not found: ID does not exist" Oct 30 19:26:55.529037 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.529007 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 19:26:55.576344 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:55.576291 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-tdnkf"] Oct 30 19:26:56.040348 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:26:56.040315 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" path="/var/lib/kubelet/pods/59d9cb1f-4b80-4bc0-97e4-7e4464117328/volumes" Oct 30 19:26:59.279597 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:26:59.279559 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:27:29.281717 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:27:29.281669 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:27:45.842013 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.841974 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:27:45.842478 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842453 5869 cpu_manager.go:414] "containerMap: removing container" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" Oct 30 19:27:45.842530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842481 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" Oct 30 19:27:45.842530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842520 5869 cpu_manager.go:414] "containerMap: removing container" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842526 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842534 5869 cpu_manager.go:414] "containerMap: removing container" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842540 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842548 5869 cpu_manager.go:414] "containerMap: removing container" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842553 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842566 5869 cpu_manager.go:414] "containerMap: removing container" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" Oct 30 19:27:45.842622 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842571 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" Oct 30 19:27:45.842822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842637 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842644 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="970d989d-e6c8-4fe9-bb41-d84a5afe0c17" containerName="istio-proxy" Oct 30 19:27:45.842822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842652 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="59d9cb1f-4b80-4bc0-97e4-7e4464117328" containerName="istio-proxy" Oct 30 19:27:45.842822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842659 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="37ed0729-22c5-46ee-a6e7-f1c8b43604bb" containerName="main" Oct 30 19:27:45.842822 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.842665 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="74284a00-9b12-4452-b0bf-684315a6b200" containerName="main" Oct 30 19:27:45.845479 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.845460 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:45.896593 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.896567 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\"" Oct 30 19:27:45.901233 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.901203 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:45.901351 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.901251 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w4lv\" (UniqueName: \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:45.943300 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.943274 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\"" Oct 30 19:27:45.943300 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.943286 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-jqbf8\"" Oct 30 19:27:45.943439 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.943300 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\"" Oct 30 19:27:45.943439 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.943299 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-fhftj\"" Oct 30 19:27:45.945105 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:45.945071 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:27:46.002230 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.002203 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:46.002344 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.002248 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w4lv\" (UniqueName: \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:46.004820 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.004799 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:46.064334 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.064307 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w4lv\" (UniqueName: \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:46.159385 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.159327 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:46.521151 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.521123 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:27:46.524445 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:27:46.524419 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ba513e_35b0_4acb_8b2a_8098432aaa9d.slice/crio-ee50d215737a11b918218c2f1cd08f179d1724b810da2dce51afa044252ae67f WatchSource:0}: Error finding container ee50d215737a11b918218c2f1cd08f179d1724b810da2dce51afa044252ae67f: Status 404 returned error can't find the container with id ee50d215737a11b918218c2f1cd08f179d1724b810da2dce51afa044252ae67f Oct 30 19:27:46.550883 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:46.550856 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" event={"ID":"96ba513e-35b0-4acb-8b2a-8098432aaa9d","Type":"ContainerStarted","Data":"ee50d215737a11b918218c2f1cd08f179d1724b810da2dce51afa044252ae67f"} Oct 30 19:27:47.556531 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:47.556491 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" event={"ID":"96ba513e-35b0-4acb-8b2a-8098432aaa9d","Type":"ContainerStarted","Data":"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d"} Oct 30 19:27:47.556964 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:47.556642 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:27:47.668274 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:27:47.668218 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podStartSLOduration=2.668201595 podStartE2EDuration="2.668201595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:47.667402212 +0000 UTC m=+2991.739226288" watchObservedRunningTime="2026-10-30 19:27:47.668201595 +0000 UTC m=+2991.740025670" Oct 30 19:27:59.281343 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:27:59.281301 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:28:18.562334 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:28:18.562301 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:28:29.281389 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:28:29.281349 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:28:59.282169 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:28:59.282129 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:29:29.283142 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:29:29.283031 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:29:58.558760 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:29:58.558721 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:29:58.562014 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:29:58.561976 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:29:59.279722 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:29:59.279683 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:30:08.562800 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:08.562751 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:30:13.586929 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:13.586881 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" containerID="cri-o://5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d" gracePeriod=45 Oct 30 19:30:18.561890 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:18.561850 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:30:18.562337 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:18.561985 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:30:28.561995 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:28.561956 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:30:29.282858 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:30:29.282818 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:30:38.561836 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:38.561790 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:30:43.838011 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:43.837985 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:30:44.006786 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.006755 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\" (UniqueName: \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\") pod \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " Oct 30 19:30:44.006930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.006828 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\" (UniqueName: \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\") pod \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\" (UID: \"96ba513e-35b0-4acb-8b2a-8098432aaa9d\") " Oct 30 19:30:44.009088 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.009061 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs" pod "96ba513e-35b0-4acb-8b2a-8098432aaa9d" (UID: "96ba513e-35b0-4acb-8b2a-8098432aaa9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:30:44.009228 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.009180 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv" pod "96ba513e-35b0-4acb-8b2a-8098432aaa9d" (UID: "96ba513e-35b0-4acb-8b2a-8098432aaa9d"). InnerVolumeSpecName "kube-api-access-6w4lv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:30:44.107746 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.107721 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w4lv\" (UniqueName: \"kubernetes.io/projected/96ba513e-35b0-4acb-8b2a-8098432aaa9d-kube-api-access-6w4lv\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:30:44.107746 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.107744 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96ba513e-35b0-4acb-8b2a-8098432aaa9d-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:30:44.342746 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.342684 5869 generic.go:356] "Generic (PLEG): container finished" podID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerID="5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d" exitCode=0 Oct 30 19:30:44.342746 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.342729 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" Oct 30 19:30:44.343238 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.342731 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" event={"ID":"96ba513e-35b0-4acb-8b2a-8098432aaa9d","Type":"ContainerDied","Data":"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d"} Oct 30 19:30:44.343238 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.342857 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2" event={"ID":"96ba513e-35b0-4acb-8b2a-8098432aaa9d","Type":"ContainerDied","Data":"ee50d215737a11b918218c2f1cd08f179d1724b810da2dce51afa044252ae67f"} Oct 30 19:30:44.343238 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.342885 5869 scope.go:122] "RemoveContainer" containerID="5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d" Oct 30 19:30:44.351225 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.351205 5869 scope.go:122] "RemoveContainer" containerID="5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d" Oct 30 19:30:44.351498 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:30:44.351480 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d\": container with ID starting with 5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d not found: ID does not exist" containerID="5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d" Oct 30 19:30:44.351545 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.351506 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d"} err="failed to get container status \"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d\": rpc error: code = NotFound desc = could not find container \"5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d\": container with ID starting with 5114f742397968259cc395c8ea975cf23968134e3075d49e9f7c1a6e9445852d not found: ID does not exist" Oct 30 19:30:44.501958 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.501934 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:30:44.549391 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:44.549368 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c959c2s2"] Oct 30 19:30:46.040230 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:30:46.040199 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" path="/var/lib/kubelet/pods/96ba513e-35b0-4acb-8b2a-8098432aaa9d/volumes" Oct 30 19:30:59.281255 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:30:59.281211 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:31:01.037748 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.037713 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:31:01.038167 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.038060 5869 cpu_manager.go:414] "containerMap: removing container" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" Oct 30 19:31:01.038167 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.038071 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" Oct 30 19:31:01.038257 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.038174 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="96ba513e-35b0-4acb-8b2a-8098432aaa9d" containerName="main" Oct 30 19:31:01.042536 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.042512 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.095689 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.095663 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:31:01.132774 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.132742 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.132906 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.132807 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxt4q\" (UniqueName: \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.141687 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.141667 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\"" Oct 30 19:31:01.141882 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.141868 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-72tvw\"" Oct 30 19:31:01.141963 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.141878 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\"" Oct 30 19:31:01.141963 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.141901 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\"" Oct 30 19:31:01.141963 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.141910 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-rk77l\"" Oct 30 19:31:01.234242 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.234221 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.234353 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.234264 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxt4q\" (UniqueName: \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.236796 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.236772 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.292680 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.292628 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxt4q\" (UniqueName: \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.355348 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.355315 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:01.718754 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:01.718731 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:31:01.721362 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:31:01.721333 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4685f062_b9e5_4969_8df0_f93215e89006.slice/crio-bd633766acda2a89941b45609467411d222b0867dc7abec14b9ce6a7cb1d6a3e WatchSource:0}: Error finding container bd633766acda2a89941b45609467411d222b0867dc7abec14b9ce6a7cb1d6a3e: Status 404 returned error can't find the container with id bd633766acda2a89941b45609467411d222b0867dc7abec14b9ce6a7cb1d6a3e Oct 30 19:31:02.432863 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:02.432828 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" event={"ID":"4685f062-b9e5-4969-8df0-f93215e89006","Type":"ContainerStarted","Data":"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c"} Oct 30 19:31:02.433323 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:02.432883 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" event={"ID":"4685f062-b9e5-4969-8df0-f93215e89006","Type":"ContainerStarted","Data":"bd633766acda2a89941b45609467411d222b0867dc7abec14b9ce6a7cb1d6a3e"} Oct 30 19:31:02.433323 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:02.432933 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:02.544352 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:02.544305 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podStartSLOduration=2.54429207 podStartE2EDuration="2.54429207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:02.543310544 +0000 UTC m=+3186.615134621" watchObservedRunningTime="2026-10-30 19:31:02.54429207 +0000 UTC m=+3186.616116146" Oct 30 19:31:29.282066 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:31:29.282026 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:31:33.438936 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:31:33.438905 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:31:59.279962 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:31:59.279921 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:32:21.801263 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.801182 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz"] Oct 30 19:32:21.804777 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.804758 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.848526 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.848499 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24"] Oct 30 19:32:21.851858 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.851842 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.896162 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.896137 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz"] Oct 30 19:32:21.902345 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902324 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902354 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902375 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902395 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd8g7\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-kube-api-access-rd8g7\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902559 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902467 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902559 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902505 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902620 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902588 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902620 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902610 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902684 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902653 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvk92\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-kube-api-access-gvk92\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902717 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902690 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902750 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902715 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902750 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902735 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575dab88-e1a1-4c6c-94b7-491b042d20c7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902835 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902814 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902870 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902855 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902875 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.902902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902891 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.902965 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902921 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:21.903000 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.902979 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:21.948503 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:21.948478 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24"] Oct 30 19:32:22.004287 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004259 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004392 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004294 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004392 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004327 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004392 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004353 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004392 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004379 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004405 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd8g7\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-kube-api-access-rd8g7\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004432 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004472 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004518 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004548 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004603 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004586 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvk92\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-kube-api-access-gvk92\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004638 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004677 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004715 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575dab88-e1a1-4c6c-94b7-491b042d20c7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004735 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004758 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004760 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004814 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004812 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004842 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004861 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.004915 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.004899 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.005414 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005045 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-certs\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.005474 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005452 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.005690 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005499 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-credential-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.005756 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005705 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-workload-socket\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.005817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005763 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575dab88-e1a1-4c6c-94b7-491b042d20c7-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.005817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.005350 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-data\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.007216 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.007193 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.007309 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.007259 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.007477 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.007456 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-envoy\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.007990 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.007970 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.060646 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.060581 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.060646 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.060594 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd8g7\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-kube-api-access-rd8g7\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.060839 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.060822 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvk92\" (UniqueName: \"kubernetes.io/projected/575dab88-e1a1-4c6c-94b7-491b042d20c7-kube-api-access-gvk92\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-clq24\" (UID: \"575dab88-e1a1-4c6c-94b7-491b042d20c7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.061285 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.061270 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c16f7d-a9ea-40bb-930b-45181394fbe7-istio-token\") pod \"openshift-ai-inference-openshift-default-5469fc87dd-gm9xz\" (UID: \"f6c16f7d-a9ea-40bb-930b-45181394fbe7\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.118412 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.118387 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:22.163830 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.163803 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:22.531716 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.531694 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz"] Oct 30 19:32:22.532950 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:32:22.532919 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c16f7d_a9ea_40bb_930b_45181394fbe7.slice/crio-4acc2b55030a5b895fe80d6ed6fd9a1e5adf96175e7393c1ee5432600096488e WatchSource:0}: Error finding container 4acc2b55030a5b895fe80d6ed6fd9a1e5adf96175e7393c1ee5432600096488e: Status 404 returned error can't find the container with id 4acc2b55030a5b895fe80d6ed6fd9a1e5adf96175e7393c1ee5432600096488e Oct 30 19:32:22.534943 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.534913 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.535002 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.534978 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.535037 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.535010 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.581408 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.581388 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24"] Oct 30 19:32:22.583552 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:32:22.583523 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575dab88_e1a1_4c6c_94b7_491b042d20c7.slice/crio-b664a7282f6bf0fd7d157d4557004e2265d8ceb4a3e6c19cd672ca2eec1e91bd WatchSource:0}: Error finding container b664a7282f6bf0fd7d157d4557004e2265d8ceb4a3e6c19cd672ca2eec1e91bd: Status 404 returned error can't find the container with id b664a7282f6bf0fd7d157d4557004e2265d8ceb4a3e6c19cd672ca2eec1e91bd Oct 30 19:32:22.585508 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.585482 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.585590 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.585537 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.585590 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.585564 5869 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Oct 30 19:32:22.790867 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.790767 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" event={"ID":"f6c16f7d-a9ea-40bb-930b-45181394fbe7","Type":"ContainerStarted","Data":"7eac241519b3af863ce0a3c4f114db5b055602d008bd962f8ffe580eb86b23bf"} Oct 30 19:32:22.790867 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.790836 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" event={"ID":"f6c16f7d-a9ea-40bb-930b-45181394fbe7","Type":"ContainerStarted","Data":"4acc2b55030a5b895fe80d6ed6fd9a1e5adf96175e7393c1ee5432600096488e"} Oct 30 19:32:22.792906 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.792877 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" event={"ID":"575dab88-e1a1-4c6c-94b7-491b042d20c7","Type":"ContainerStarted","Data":"89737cb9a0ee1bcee9993cd2e6be92e7110d7ca999843714eabd08766dcb8b43"} Oct 30 19:32:22.792906 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.792921 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" event={"ID":"575dab88-e1a1-4c6c-94b7-491b042d20c7","Type":"ContainerStarted","Data":"b664a7282f6bf0fd7d157d4557004e2265d8ceb4a3e6c19cd672ca2eec1e91bd"} Oct 30 19:32:22.907568 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:22.907513 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podStartSLOduration=1.9074973929999999 podStartE2EDuration="1.907497393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:22.906519923 +0000 UTC m=+3266.978343999" watchObservedRunningTime="2026-10-30 19:32:22.907497393 +0000 UTC m=+3266.979321469" Oct 30 19:32:23.025137 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.025052 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podStartSLOduration=2.025033152 podStartE2EDuration="2.025033152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:23.023618963 +0000 UTC m=+3267.095443041" watchObservedRunningTime="2026-10-30 19:32:23.025033152 +0000 UTC m=+3267.096857231" Oct 30 19:32:23.118820 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.118724 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:23.120198 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.120165 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:23.120340 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.120211 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:23.164784 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.164754 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:23.165868 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.165845 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:23.165949 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:23.165886 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:24.119521 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:24.119489 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:24.120044 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:24.119535 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:24.164359 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:24.164330 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:24.164488 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:24.164373 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:25.119548 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:25.119503 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:25.119921 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:25.119571 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:25.164951 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:25.164923 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:25.165076 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:25.164974 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:26.119568 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:26.119525 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:26.120129 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:26.119578 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:26.164820 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:26.164790 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:26.164937 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:26.164836 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:27.120539 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:27.120495 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:27.120962 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:27.120561 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:27.164650 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:27.164622 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:27.164764 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:27.164665 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:28.119313 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:28.119280 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Oct 30 19:32:28.119509 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:28.119332 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" podUID="f6c16f7d-a9ea-40bb-930b-45181394fbe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Oct 30 19:32:28.164691 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:28.164656 5869 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-5469fc87dd-clq24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Oct 30 19:32:28.165045 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:28.164709 5869 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" podUID="575dab88-e1a1-4c6c-94b7-491b042d20c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Oct 30 19:32:29.122630 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.122598 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:29.122896 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.122874 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:29.123492 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.123471 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-gm9xz" Oct 30 19:32:29.168579 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.168555 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:29.168951 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.168814 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:29.169517 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:32:29.169499 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-5469fc87dd-clq24" Oct 30 19:32:29.280104 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:32:29.280060 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:32:59.279750 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:32:59.279709 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:33:13.542344 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:13.542304 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:33:23.438791 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:23.438742 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:28.563534 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:28.563486 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" containerID="cri-o://0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c" gracePeriod=45 Oct 30 19:33:29.279015 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:33:29.278980 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:33:33.438922 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:33.438869 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:43.438805 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:43.438757 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:43.439280 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:43.438879 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:33:53.439164 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:53.439086 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:33:58.797267 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:58.797241 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:33:58.917962 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:58.917891 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\" (UniqueName: \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\") pod \"4685f062-b9e5-4969-8df0-f93215e89006\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " Oct 30 19:33:58.917962 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:58.917941 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\" (UniqueName: \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\") pod \"4685f062-b9e5-4969-8df0-f93215e89006\" (UID: \"4685f062-b9e5-4969-8df0-f93215e89006\") " Oct 30 19:33:58.920255 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:58.920229 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q" pod "4685f062-b9e5-4969-8df0-f93215e89006" (UID: "4685f062-b9e5-4969-8df0-f93215e89006"). InnerVolumeSpecName "kube-api-access-xxt4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:33:58.920255 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:58.920235 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs" pod "4685f062-b9e5-4969-8df0-f93215e89006" (UID: "4685f062-b9e5-4969-8df0-f93215e89006"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:33:59.019220 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.019191 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685f062-b9e5-4969-8df0-f93215e89006-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:59.019220 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.019218 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxt4q\" (UniqueName: \"kubernetes.io/projected/4685f062-b9e5-4969-8df0-f93215e89006-kube-api-access-xxt4q\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:33:59.239797 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.239769 5869 generic.go:356] "Generic (PLEG): container finished" podID="4685f062-b9e5-4969-8df0-f93215e89006" containerID="0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c" exitCode=0 Oct 30 19:33:59.239918 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.239817 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" Oct 30 19:33:59.239918 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.239826 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" event={"ID":"4685f062-b9e5-4969-8df0-f93215e89006","Type":"ContainerDied","Data":"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c"} Oct 30 19:33:59.239918 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.239867 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk" event={"ID":"4685f062-b9e5-4969-8df0-f93215e89006","Type":"ContainerDied","Data":"bd633766acda2a89941b45609467411d222b0867dc7abec14b9ce6a7cb1d6a3e"} Oct 30 19:33:59.239918 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.239888 5869 scope.go:122] "RemoveContainer" containerID="0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c" Oct 30 19:33:59.249317 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.249297 5869 scope.go:122] "RemoveContainer" containerID="0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c" Oct 30 19:33:59.249565 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:33:59.249547 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c\": container with ID starting with 0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c not found: ID does not exist" containerID="0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c" Oct 30 19:33:59.249610 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.249583 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c"} err="failed to get container status \"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c\": rpc error: code = NotFound desc = could not find container \"0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c\": container with ID starting with 0792ddc4c66b1839d5746be6b5793c8fe4e2e227ce3654240729d81bcf89930c not found: ID does not exist" Oct 30 19:33:59.280480 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:33:59.280450 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:33:59.398523 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.398447 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:33:59.445721 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:33:59.445695 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d46nlk"] Oct 30 19:34:00.042696 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:34:00.042662 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4685f062-b9e5-4969-8df0-f93215e89006" path="/var/lib/kubelet/pods/4685f062-b9e5-4969-8df0-f93215e89006/volumes" Oct 30 19:34:29.280891 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:34:29.280853 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:34:59.279942 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:34:59.279907 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:35:29.281883 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:35:29.281798 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:35:59.280871 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:35:59.280831 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:36:29.281400 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:36:29.281358 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:36:48.829529 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.829497 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:36:48.829958 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.829905 5869 cpu_manager.go:414] "containerMap: removing container" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" Oct 30 19:36:48.829958 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.829918 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" Oct 30 19:36:48.830044 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.829982 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="4685f062-b9e5-4969-8df0-f93215e89006" containerName="main" Oct 30 19:36:48.833106 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.833081 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.887169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.887141 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:36:48.915930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.915896 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.915930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.915934 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhf46\" (UniqueName: \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.916169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.915959 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.916169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.916015 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.916169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.916067 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.916169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.916083 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:48.932733 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.932667 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"openshift-service-ca.crt\"" Oct 30 19:36:48.932733 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.932691 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"default-dockercfg-z6ldg\"" Oct 30 19:36:48.932733 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.932692 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"kube-root-ca.crt\"" Oct 30 19:36:48.933233 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:48.932845 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"model-routing-only-7aad4d71-kserve-self-signed-certs\"" Oct 30 19:36:49.017553 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017522 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017700 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017559 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhf46\" (UniqueName: \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017700 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017585 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017711 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017760 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017817 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017785 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.017982 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.017945 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.018053 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.018033 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.018053 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.018043 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.020240 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.020219 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.020497 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.020481 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.074239 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.074209 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhf46\" (UniqueName: \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-w44s8\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.147116 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.147029 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:49.511466 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:49.511445 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:36:49.513782 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:36:49.513753 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45ffae7_15af_4782_a8ab_09015cfba1ee.slice/crio-9f13c433c82ec89da953bbf6c34b462bbf9e5207a80c03972242ed650022a5be WatchSource:0}: Error finding container 9f13c433c82ec89da953bbf6c34b462bbf9e5207a80c03972242ed650022a5be: Status 404 returned error can't find the container with id 9f13c433c82ec89da953bbf6c34b462bbf9e5207a80c03972242ed650022a5be Oct 30 19:36:50.032148 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:50.032112 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" event={"ID":"e45ffae7-15af-4782-a8ab-09015cfba1ee","Type":"ContainerStarted","Data":"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782"} Oct 30 19:36:50.032593 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:50.032164 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" event={"ID":"e45ffae7-15af-4782-a8ab-09015cfba1ee","Type":"ContainerStarted","Data":"9f13c433c82ec89da953bbf6c34b462bbf9e5207a80c03972242ed650022a5be"} Oct 30 19:36:50.142969 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:50.142928 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" podStartSLOduration=2.1429147889999998 podStartE2EDuration="2.142914789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:50.142337507 +0000 UTC m=+3534.214161574" watchObservedRunningTime="2026-10-30 19:36:50.142914789 +0000 UTC m=+3534.214738865" Oct 30 19:36:50.147913 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:50.147885 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:59.148169 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:59.148137 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:59.160110 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:36:59.160070 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:36:59.280209 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:36:59.280175 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:37:00.084639 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:37:00.084611 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:37:29.279437 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:37:29.279395 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:37:59.279516 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:37:59.279479 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:38:08.583735 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.583699 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:38:08.593834 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:38:08.593795 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" cmd=["/bin/sleep","15"] Oct 30 19:38:08.593970 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.593855 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:38:08.593970 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:38:08.593868 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerName="main" containerID="cri-o://a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" Oct 30 19:38:08.593970 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.593921 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerName="main" containerID="cri-o://a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" gracePeriod=60 Oct 30 19:38:08.940395 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.940372 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:38:08.972347 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972323 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972473 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972370 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\" (UniqueName: \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972473 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972404 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972473 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972442 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972592 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972520 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972592 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972561 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\" (UniqueName: \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\") pod \"e45ffae7-15af-4782-a8ab-09015cfba1ee\" (UID: \"e45ffae7-15af-4782-a8ab-09015cfba1ee\") " Oct 30 19:38:08.972687 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972601 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:38:08.972754 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972708 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:38:08.972754 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972735 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:38:08.972925 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972909 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:08.972978 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972926 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:08.972978 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.972937 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:08.974804 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.974773 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:08.974804 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.974790 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "kube-api-access-nhf46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:08.974804 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:08.974792 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm" pod "e45ffae7-15af-4782-a8ab-09015cfba1ee" (UID: "e45ffae7-15af-4782-a8ab-09015cfba1ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:38:09.073788 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.073764 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45ffae7-15af-4782-a8ab-09015cfba1ee-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:09.073921 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.073826 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45ffae7-15af-4782-a8ab-09015cfba1ee-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:09.073921 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.073842 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhf46\" (UniqueName: \"kubernetes.io/projected/e45ffae7-15af-4782-a8ab-09015cfba1ee-kube-api-access-nhf46\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:38:09.390412 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.390373 5869 generic.go:356] "Generic (PLEG): container finished" podID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerID="a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" exitCode=0 Oct 30 19:38:09.390577 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.390435 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" Oct 30 19:38:09.390577 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.390478 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" event={"ID":"e45ffae7-15af-4782-a8ab-09015cfba1ee","Type":"ContainerDied","Data":"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782"} Oct 30 19:38:09.390577 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.390537 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8" event={"ID":"e45ffae7-15af-4782-a8ab-09015cfba1ee","Type":"ContainerDied","Data":"9f13c433c82ec89da953bbf6c34b462bbf9e5207a80c03972242ed650022a5be"} Oct 30 19:38:09.390577 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.390560 5869 scope.go:122] "RemoveContainer" containerID="a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" Oct 30 19:38:09.401055 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.401037 5869 scope.go:122] "RemoveContainer" containerID="a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" Oct 30 19:38:09.401363 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:38:09.401344 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782\": container with ID starting with a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782 not found: ID does not exist" containerID="a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782" Oct 30 19:38:09.401434 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.401372 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782"} err="failed to get container status \"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782\": rpc error: code = NotFound desc = could not find container \"a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782\": container with ID starting with a6e704eb23fc0c45e7f764af9434eeb433934788a62e1fa947e2327ad1c4c782 not found: ID does not exist" Oct 30 19:38:09.549710 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.549681 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:38:09.555883 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:09.555859 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-w44s8"] Oct 30 19:38:10.042018 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:10.041988 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" path="/var/lib/kubelet/pods/e45ffae7-15af-4782-a8ab-09015cfba1ee/volumes" Oct 30 19:38:16.548798 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.548768 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:38:16.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.549165 5869 cpu_manager.go:414] "containerMap: removing container" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerName="main" Oct 30 19:38:16.549211 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.549176 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerName="main" Oct 30 19:38:16.549292 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.549247 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45ffae7-15af-4782-a8ab-09015cfba1ee" containerName="main" Oct 30 19:38:16.553801 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.553781 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638464 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638430 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638617 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638480 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638617 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638605 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638735 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638645 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638735 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638672 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.638735 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.638696 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knmcx\" (UniqueName: \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.652380 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.652352 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"kube-root-ca.crt\"" Oct 30 19:38:16.652750 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.652730 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"openshift-service-ca.crt\"" Oct 30 19:38:16.652857 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.652729 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"model-routing-only-7aad4d71-kserve-self-signed-certs\"" Oct 30 19:38:16.652857 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.652784 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"default-dockercfg-7r2gn\"" Oct 30 19:38:16.653321 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.653305 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:38:16.740161 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740135 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740274 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740171 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740274 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740217 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740274 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740245 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740274 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740269 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740485 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740293 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knmcx\" (UniqueName: \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740610 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740585 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740610 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740604 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.740732 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.740646 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.742729 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.742711 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.742950 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.742933 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.796451 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.796425 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knmcx\" (UniqueName: \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:16.866601 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:16.866548 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:17.241319 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:17.241283 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:38:17.429234 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:17.429196 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" event={"ID":"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963","Type":"ContainerStarted","Data":"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92"} Oct 30 19:38:17.429399 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:17.429289 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" event={"ID":"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963","Type":"ContainerStarted","Data":"f95b857207dc6d3ae8f80963c79b3696952310caa89cf1367f694dab3576d3f3"} Oct 30 19:38:17.541202 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:17.541075 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" podStartSLOduration=1.541061622 podStartE2EDuration="1.541061622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:17.540566188 +0000 UTC m=+3621.612390265" watchObservedRunningTime="2026-10-30 19:38:17.541061622 +0000 UTC m=+3621.612885698" Oct 30 19:38:17.867162 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:17.867065 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:26.867345 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:26.867304 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:26.879738 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:26.879694 5869 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:27.485895 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:38:27.485864 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:38:29.279492 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:38:29.279450 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:38:59.280900 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:38:59.280858 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:39:29.279723 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:39:29.279686 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:39:35.962663 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:35.962626 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:39:35.972722 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:39:35.972671 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" cmd=["/bin/sleep","15"] Oct 30 19:39:35.972872 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:35.972738 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:39:35.972872 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:39:35.972752 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" containerID="cri-o://8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" Oct 30 19:39:35.972872 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:35.972788 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" containerID="cri-o://8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" gracePeriod=60 Oct 30 19:39:36.475446 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.475407 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused" Oct 30 19:39:36.719385 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.719362 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:39:36.779023 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.778997 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779168 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779062 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779168 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779161 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\" (UniqueName: \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779250 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779190 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779307 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779255 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779363 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779317 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\" (UniqueName: \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\") pod \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\" (UID: \"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963\") " Oct 30 19:39:36.779363 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779331 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:39:36.779530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779503 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:39:36.779530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779522 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:39:36.779686 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779670 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.779738 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779693 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.779738 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.779708 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.781615 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.781589 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:39:36.781709 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.781665 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:39:36.781709 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.781682 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx" pod "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" (UID: "7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963"). InnerVolumeSpecName "kube-api-access-knmcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:39:36.784738 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.784707 5869 generic.go:356] "Generic (PLEG): container finished" podID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerID="8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" exitCode=0 Oct 30 19:39:36.784831 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.784789 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" Oct 30 19:39:36.784879 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.784826 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" event={"ID":"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963","Type":"ContainerDied","Data":"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92"} Oct 30 19:39:36.784920 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.784905 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr" event={"ID":"7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963","Type":"ContainerDied","Data":"f95b857207dc6d3ae8f80963c79b3696952310caa89cf1367f694dab3576d3f3"} Oct 30 19:39:36.784958 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.784927 5869 scope.go:122] "RemoveContainer" containerID="8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" Oct 30 19:39:36.799445 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.799423 5869 scope.go:122] "RemoveContainer" containerID="8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" Oct 30 19:39:36.799696 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:39:36.799680 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92\": container with ID starting with 8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92 not found: ID does not exist" containerID="8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92" Oct 30 19:39:36.799753 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.799702 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92"} err="failed to get container status \"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92\": rpc error: code = NotFound desc = could not find container \"8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92\": container with ID starting with 8e533375ed931feab1cc3e31122296e2ee2eca6d1bc7f3215456723ca3b3fa92 not found: ID does not exist" Oct 30 19:39:36.880599 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.880573 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knmcx\" (UniqueName: \"kubernetes.io/projected/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-kube-api-access-knmcx\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.880599 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.880598 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.880776 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.880609 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:39:36.938671 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.938645 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:39:36.943167 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:36.943144 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-7aad4d71-kserve-6557db66dd-jmkhr"] Oct 30 19:39:38.041245 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:38.041209 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" path="/var/lib/kubelet/pods/7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963/volumes" Oct 30 19:39:45.823056 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.823014 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:39:45.823521 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.823437 5869 cpu_manager.go:414] "containerMap: removing container" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" Oct 30 19:39:45.823521 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.823450 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" Oct 30 19:39:45.823604 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.823535 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fd2de5e-ebaf-4a7f-a7aa-b0007b9f5963" containerName="main" Oct 30 19:39:45.827920 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.827897 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:45.859804 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.859771 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jvrv\" (UniqueName: \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:45.859995 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.859857 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:45.883872 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.883847 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:39:45.930054 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.930029 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\"" Oct 30 19:39:45.930930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.930905 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-epp-sa-dockercfg-p5v4f\"" Oct 30 19:39:45.930930 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.930915 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\"" Oct 30 19:39:45.931069 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.930929 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\"" Oct 30 19:39:45.931069 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.930948 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-m94j9\"" Oct 30 19:39:45.960604 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.960578 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jvrv\" (UniqueName: \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:45.960704 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.960636 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:45.963208 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:45.963188 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:46.016957 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.016927 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jvrv\" (UniqueName: \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:46.141649 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.141591 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:46.513437 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.513416 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:39:46.515718 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:39:46.515686 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69225a07_3b94_4964_b27a_3cfc7bee1c15.slice/crio-9c291e6cd0e694dd17422c4182f3300df22a1f8374619acea7ca3170af50ae4f WatchSource:0}: Error finding container 9c291e6cd0e694dd17422c4182f3300df22a1f8374619acea7ca3170af50ae4f: Status 404 returned error can't find the container with id 9c291e6cd0e694dd17422c4182f3300df22a1f8374619acea7ca3170af50ae4f Oct 30 19:39:46.877387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.877300 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" event={"ID":"69225a07-3b94-4964-b27a-3cfc7bee1c15","Type":"ContainerStarted","Data":"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2"} Oct 30 19:39:46.877387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.877356 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" event={"ID":"69225a07-3b94-4964-b27a-3cfc7bee1c15","Type":"ContainerStarted","Data":"9c291e6cd0e694dd17422c4182f3300df22a1f8374619acea7ca3170af50ae4f"} Oct 30 19:39:46.877387 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.877381 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:39:46.989745 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:39:46.989698 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podStartSLOduration=1.98968421 podStartE2EDuration="1.98968421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:46.988041376 +0000 UTC m=+3711.059865452" watchObservedRunningTime="2026-10-30 19:39:46.98968421 +0000 UTC m=+3711.061508278" Oct 30 19:39:59.280024 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:39:59.279989 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:40:17.883116 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:40:17.883065 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:40:29.280019 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:40:29.279980 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:40:59.279252 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:40:59.279216 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:41:29.282068 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:41:29.281973 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:41:59.280612 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:41:59.280570 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:42:29.281775 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:42:29.281737 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:42:40.825479 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:42:40.825447 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:42:47.883331 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:42:47.883290 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:42:55.845792 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:42:55.845751 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" containerID="cri-o://9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2" gracePeriod=45 Oct 30 19:42:57.883999 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:42:57.883958 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:42:59.283030 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:42:59.282990 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:43:07.883629 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:07.883579 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:43:07.884165 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:07.883734 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:43:17.883984 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:17.883946 5869 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:43:26.089950 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.089926 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:43:26.271200 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.271174 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\" (UniqueName: \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\") pod \"69225a07-3b94-4964-b27a-3cfc7bee1c15\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " Oct 30 19:43:26.271330 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.271223 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\" (UniqueName: \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\") pod \"69225a07-3b94-4964-b27a-3cfc7bee1c15\" (UID: \"69225a07-3b94-4964-b27a-3cfc7bee1c15\") " Oct 30 19:43:26.273510 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.273478 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs" pod "69225a07-3b94-4964-b27a-3cfc7bee1c15" (UID: "69225a07-3b94-4964-b27a-3cfc7bee1c15"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:43:26.273624 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.273573 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv" pod "69225a07-3b94-4964-b27a-3cfc7bee1c15" (UID: "69225a07-3b94-4964-b27a-3cfc7bee1c15"). InnerVolumeSpecName "kube-api-access-9jvrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:43:26.372039 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.372013 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69225a07-3b94-4964-b27a-3cfc7bee1c15-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:26.372039 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.372036 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jvrv\" (UniqueName: \"kubernetes.io/projected/69225a07-3b94-4964-b27a-3cfc7bee1c15-kube-api-access-9jvrv\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:43:26.862409 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.862370 5869 generic.go:356] "Generic (PLEG): container finished" podID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerID="9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2" exitCode=0 Oct 30 19:43:26.862644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.862426 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" Oct 30 19:43:26.862644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.862453 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" event={"ID":"69225a07-3b94-4964-b27a-3cfc7bee1c15","Type":"ContainerDied","Data":"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2"} Oct 30 19:43:26.862644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.862510 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9" event={"ID":"69225a07-3b94-4964-b27a-3cfc7bee1c15","Type":"ContainerDied","Data":"9c291e6cd0e694dd17422c4182f3300df22a1f8374619acea7ca3170af50ae4f"} Oct 30 19:43:26.862644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.862533 5869 scope.go:122] "RemoveContainer" containerID="9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2" Oct 30 19:43:26.871916 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.871897 5869 scope.go:122] "RemoveContainer" containerID="9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2" Oct 30 19:43:26.872210 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:43:26.872188 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2\": container with ID starting with 9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2 not found: ID does not exist" containerID="9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2" Oct 30 19:43:26.872282 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:26.872219 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2"} err="failed to get container status \"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2\": rpc error: code = NotFound desc = could not find container \"9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2\": container with ID starting with 9cfdda98e9557f4c131c1b72bd25e79a8dfe9bacc575f30fa3162ec5e529fbd2 not found: ID does not exist" Oct 30 19:43:27.021060 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:27.021036 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:43:27.026236 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:27.026212 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-2nsh9"] Oct 30 19:43:28.040557 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:43:28.040521 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" path="/var/lib/kubelet/pods/69225a07-3b94-4964-b27a-3cfc7bee1c15/volumes" Oct 30 19:43:29.279929 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:43:29.279891 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:43:59.281687 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:43:59.281647 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:44:29.280330 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:29.280215 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:44:51.410001 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.409961 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:51.410556 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.410426 5869 cpu_manager.go:414] "containerMap: removing container" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" Oct 30 19:44:51.410556 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.410446 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" Oct 30 19:44:51.410556 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.410532 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="69225a07-3b94-4964-b27a-3cfc7bee1c15" containerName="main" Oct 30 19:44:51.414063 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.414044 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.514705 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.514678 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\"" Oct 30 19:44:51.514853 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.514753 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\"" Oct 30 19:44:51.514853 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.514753 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-x7hxk\"" Oct 30 19:44:51.515038 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.515025 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\"" Oct 30 19:44:51.515868 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.515850 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:51.549727 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549703 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsxh4\" (UniqueName: \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.549840 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549740 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.549840 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549759 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.549840 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549821 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.549944 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549862 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.549979 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.549952 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650566 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650539 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650672 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650572 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650672 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650588 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650672 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650619 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650827 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650682 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsxh4\" (UniqueName: \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.650827 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650711 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.651020 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.650997 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.651086 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.651030 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.651191 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.651172 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.653143 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.653124 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.653414 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.653393 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.707357 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.707336 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsxh4\" (UniqueName: \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\") pod \"rolling-upgrade-test-kserve-78f4d666cb-mc2q7\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:51.727720 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:51.727701 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:52.092798 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:52.092772 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:52.094694 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:44:52.094662 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd475f69_f207_43c5_8daa_c8695225b523.slice/crio-45f9c153bb751a8d332ffd4381499514abd76fc3f4cd415d7d64fbb4843f0072 WatchSource:0}: Error finding container 45f9c153bb751a8d332ffd4381499514abd76fc3f4cd415d7d64fbb4843f0072: Status 404 returned error can't find the container with id 45f9c153bb751a8d332ffd4381499514abd76fc3f4cd415d7d64fbb4843f0072 Oct 30 19:44:52.288670 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:52.288628 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" event={"ID":"bd475f69-f207-43c5-8daa-c8695225b523","Type":"ContainerStarted","Data":"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b"} Oct 30 19:44:52.288850 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:52.288680 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" event={"ID":"bd475f69-f207-43c5-8daa-c8695225b523","Type":"ContainerStarted","Data":"45f9c153bb751a8d332ffd4381499514abd76fc3f4cd415d7d64fbb4843f0072"} Oct 30 19:44:52.407353 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:52.407270 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" podStartSLOduration=1.407256432 podStartE2EDuration="1.407256432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:52.407143405 +0000 UTC m=+4016.478967479" watchObservedRunningTime="2026-10-30 19:44:52.407256432 +0000 UTC m=+4016.479080557" Oct 30 19:44:52.728423 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:52.728398 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:54.427761 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:54.427726 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:54.473364 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:54.473333 5869 secret.go:189] Couldn't get secret e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-self-signed-certs: secret "rolling-upgrade-test-kserve-self-signed-certs" not found Oct 30 19:44:54.473515 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:54.473444 5869 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs podName:bd475f69-f207-43c5-8daa-c8695225b523 nodeName:}" failed. No retries permitted until 2026-10-30 19:44:54.973425344 +0000 UTC m=+4019.045249411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs") pod "rolling-upgrade-test-kserve-78f4d666cb-mc2q7" (UID: "bd475f69-f207-43c5-8daa-c8695225b523") : secret "rolling-upgrade-test-kserve-self-signed-certs" not found Oct 30 19:44:54.980296 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:54.980269 5869 secret.go:189] Couldn't get secret e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-self-signed-certs: secret "rolling-upgrade-test-kserve-self-signed-certs" not found Oct 30 19:44:54.980469 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:54.980330 5869 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs podName:bd475f69-f207-43c5-8daa-c8695225b523 nodeName:}" failed. No retries permitted until 2026-10-30 19:44:55.980316861 +0000 UTC m=+4020.052140929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs") pod "rolling-upgrade-test-kserve-78f4d666cb-mc2q7" (UID: "bd475f69-f207-43c5-8daa-c8695225b523") : secret "rolling-upgrade-test-kserve-self-signed-certs" not found Oct 30 19:44:55.312058 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:55.311975 5869 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" cmd=["/bin/sleep","15"] Oct 30 19:44:55.312058 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.312031 5869 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" Oct 30 19:44:55.312058 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:55.312043 5869 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" podUID="bd475f69-f207-43c5-8daa-c8695225b523" containerName="main" containerID="cri-o://a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" Oct 30 19:44:55.312286 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.312104 5869 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" podUID="bd475f69-f207-43c5-8daa-c8695225b523" containerName="main" containerID="cri-o://a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" gracePeriod=60 Oct 30 19:44:55.560448 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.560424 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:55.685793 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685767 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.685945 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685803 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\" (UniqueName: \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.685945 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685851 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.685945 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685889 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\" (UniqueName: \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.685945 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685921 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.686205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685964 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:55.686205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.685998 5869 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\") pod \"bd475f69-f207-43c5-8daa-c8695225b523\" (UID: \"bd475f69-f207-43c5-8daa-c8695225b523\") " Oct 30 19:44:55.686205 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.686179 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:55.686365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.686308 5869 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-home\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:55.686365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.686316 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:55.686365 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.686325 5869 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-tmp-dir\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:55.688225 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.688202 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "kube-api-access-lsxh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:44:55.688339 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.688237 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:44:55.688339 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.688278 5869 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm" pod "bd475f69-f207-43c5-8daa-c8695225b523" (UID: "bd475f69-f207-43c5-8daa-c8695225b523"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:44:55.787297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.787273 5869 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd475f69-f207-43c5-8daa-c8695225b523-tls-certs\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:55.787297 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.787295 5869 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-dshm\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:55.787435 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.787304 5869 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd475f69-f207-43c5-8daa-c8695225b523-model-cache\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:55.787435 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:55.787313 5869 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsxh4\" (UniqueName: \"kubernetes.io/projected/bd475f69-f207-43c5-8daa-c8695225b523-kube-api-access-lsxh4\") on node \"ip-10-0-128-38.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:44:56.311320 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.311247 5869 generic.go:356] "Generic (PLEG): container finished" podID="bd475f69-f207-43c5-8daa-c8695225b523" containerID="a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" exitCode=0 Oct 30 19:44:56.311320 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.311300 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" event={"ID":"bd475f69-f207-43c5-8daa-c8695225b523","Type":"ContainerDied","Data":"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b"} Oct 30 19:44:56.311320 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.311307 5869 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" Oct 30 19:44:56.311532 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.311362 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7" event={"ID":"bd475f69-f207-43c5-8daa-c8695225b523","Type":"ContainerDied","Data":"45f9c153bb751a8d332ffd4381499514abd76fc3f4cd415d7d64fbb4843f0072"} Oct 30 19:44:56.311532 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.311384 5869 scope.go:122] "RemoveContainer" containerID="a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" Oct 30 19:44:56.319976 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.319950 5869 scope.go:122] "RemoveContainer" containerID="a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" Oct 30 19:44:56.320324 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:56.320304 5869 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b\": container with ID starting with a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b not found: ID does not exist" containerID="a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b" Oct 30 19:44:56.320389 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.320330 5869 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b"} err="failed to get container status \"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b\": rpc error: code = NotFound desc = could not find container \"a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b\": container with ID starting with a9cb50db3e6697dad900a24351474f9e7fd74ea1ff3fbf6457d5bdee2b62e92b not found: ID does not exist" Oct 30 19:44:56.470433 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.470410 5869 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:56.517591 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:56.517564 5869 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-mc2q7"] Oct 30 19:44:58.041914 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:44:58.041872 5869 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd475f69-f207-43c5-8daa-c8695225b523" path="/var/lib/kubelet/pods/bd475f69-f207-43c5-8daa-c8695225b523/volumes" Oct 30 19:44:59.280870 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:44:59.280833 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:45:29.281256 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:45:29.281221 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:45:59.283033 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:45:59.282988 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:46:29.280239 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:46:29.280191 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:46:59.280632 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:46:59.280588 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:47:03.051982 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:03.051947 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-2dzfn_bfb6eabc-9dde-46b5-bcae-cf775eaeed51/istio-proxy/0.log" Oct 30 19:47:04.098844 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:04.098803 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-clq24_575dab88-e1a1-4c6c-94b7-491b042d20c7/istio-proxy/0.log" Oct 30 19:47:04.303722 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:04.303692 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-gm9xz_f6c16f7d-a9ea-40bb-930b-45181394fbe7/istio-proxy/0.log" Oct 30 19:47:08.882072 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:08.881991 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-2dzfn_bfb6eabc-9dde-46b5-bcae-cf775eaeed51/istio-proxy/0.log" Oct 30 19:47:09.756644 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:09.756606 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-clq24_575dab88-e1a1-4c6c-94b7-491b042d20c7/istio-proxy/0.log" Oct 30 19:47:09.970115 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:09.970062 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-5469fc87dd-gm9xz_f6c16f7d-a9ea-40bb-930b-45181394fbe7/istio-proxy/0.log" Oct 30 19:47:14.192308 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:14.192281 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-z67lc_d28ef0aa-fac8-418d-a1bd-cb9967713197/authorino/0.log" Oct 30 19:47:14.712776 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:14.712741 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-bm9bl_eed635c9-5674-4c53-93a0-9d0fd88eabfd/manager/0.log" Oct 30 19:47:15.815809 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:15.815771 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-grd5p_08866c11-f350-4e1b-9548-629d7e67f0a0/limitador/0.log" Oct 30 19:47:29.281000 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:47:29.280953 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:47:31.469902 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:31.469871 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-plzrc_182b13f1-6909-4251-8455-eed4d52ae204/global-pull-secret-syncer/0.log" Oct 30 19:47:34.395864 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:34.395826 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zllgp_7268393d-9be4-4e1c-87ff-c9ac7cfd7072/konnectivity-agent/0.log" Oct 30 19:47:34.756244 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:34.756210 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-38.us-west-2.compute.internal_1c6479c60b669fe79980e9a61328bda9/haproxy/0.log" Oct 30 19:47:39.558987 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:39.558955 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-z67lc_d28ef0aa-fac8-418d-a1bd-cb9967713197/authorino/0.log" Oct 30 19:47:40.158594 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:40.158564 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-bm9bl_eed635c9-5674-4c53-93a0-9d0fd88eabfd/manager/0.log" Oct 30 19:47:41.903885 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:41.903853 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-grd5p_08866c11-f350-4e1b-9548-629d7e67f0a0/limitador/0.log" Oct 30 19:47:42.483041 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.483002 5869 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5"] Oct 30 19:47:42.483426 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.483411 5869 cpu_manager.go:414] "containerMap: removing container" podUID="bd475f69-f207-43c5-8daa-c8695225b523" containerName="main" Oct 30 19:47:42.483483 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.483427 5869 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd475f69-f207-43c5-8daa-c8695225b523" containerName="main" Oct 30 19:47:42.483526 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.483514 5869 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd475f69-f207-43c5-8daa-c8695225b523" containerName="main" Oct 30 19:47:42.486722 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.486699 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.541653 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.541629 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5"] Oct 30 19:47:42.587561 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.587538 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ndc8\"/\"kube-root-ca.crt\"" Oct 30 19:47:42.587561 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.587538 5869 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ndc8\"/\"openshift-service-ca.crt\"" Oct 30 19:47:42.587724 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.587587 5869 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ndc8\"/\"default-dockercfg-xx74q\"" Oct 30 19:47:42.638917 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.638889 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtxm9\" (UniqueName: \"kubernetes.io/projected/923c56fd-a792-465b-9394-c9941c030b45-kube-api-access-rtxm9\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.639053 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.639002 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-lib-modules\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.639144 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.639070 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-proc\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.639144 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.639127 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-sys\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.639271 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.639228 5869 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-podres\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740559 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740504 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-lib-modules\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740559 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740540 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-proc\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740561 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-sys\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740595 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-podres\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740618 5869 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtxm9\" (UniqueName: \"kubernetes.io/projected/923c56fd-a792-465b-9394-c9941c030b45-kube-api-access-rtxm9\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740638 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-proc\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740648 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-lib-modules\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740648 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-sys\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.740749 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.740713 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/923c56fd-a792-465b-9394-c9941c030b45-podres\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.797530 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.797510 5869 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtxm9\" (UniqueName: \"kubernetes.io/projected/923c56fd-a792-465b-9394-c9941c030b45-kube-api-access-rtxm9\") pod \"perf-node-gather-daemonset-69jv5\" (UID: \"923c56fd-a792-465b-9394-c9941c030b45\") " pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:42.799397 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:42.799364 5869 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:43.165138 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:43.165109 5869 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5"] Oct 30 19:47:43.167768 ip-10-0-128-38 kubenswrapper[5869]: W1030 19:47:43.167737 5869 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod923c56fd_a792_465b_9394_c9941c030b45.slice/crio-61cb06b9d2019e11729efca0749bd4afb3d5f377481563439790aff38774d547 WatchSource:0}: Error finding container 61cb06b9d2019e11729efca0749bd4afb3d5f377481563439790aff38774d547: Status 404 returned error can't find the container with id 61cb06b9d2019e11729efca0749bd4afb3d5f377481563439790aff38774d547 Oct 30 19:47:44.087772 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:44.087724 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" event={"ID":"923c56fd-a792-465b-9394-c9941c030b45","Type":"ContainerStarted","Data":"bb59ee54d79a132328f54dae5a3e4e02d8a204fff909e3d8f79c846c7de9f6fb"} Oct 30 19:47:44.087772 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:44.087779 5869 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" event={"ID":"923c56fd-a792-465b-9394-c9941c030b45","Type":"ContainerStarted","Data":"61cb06b9d2019e11729efca0749bd4afb3d5f377481563439790aff38774d547"} Oct 30 19:47:44.088005 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:44.087814 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:44.198576 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:44.198530 5869 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" podStartSLOduration=2.198517947 podStartE2EDuration="2.198517947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:44.197790524 +0000 UTC m=+4188.269614602" watchObservedRunningTime="2026-10-30 19:47:44.198517947 +0000 UTC m=+4188.270342023" Oct 30 19:47:44.725152 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:44.725118 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/alertmanager/0.log" Oct 30 19:47:45.115414 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:45.115333 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/config-reloader/0.log" Oct 30 19:47:45.504723 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:45.504697 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/kube-rbac-proxy-web/0.log" Oct 30 19:47:45.893802 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:45.893733 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/kube-rbac-proxy/0.log" Oct 30 19:47:46.284134 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:46.284074 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/kube-rbac-proxy-metric/0.log" Oct 30 19:47:46.674473 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:46.674396 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/prom-label-proxy/0.log" Oct 30 19:47:47.068213 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:47.068190 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a693e16c-af01-4ddd-b008-a1cf5cf99a4d/init-config-reloader/0.log" Oct 30 19:47:47.923969 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:47.923934 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qc4zv_79e792e9-28c1-4127-b364-3679fcf27c7c/kube-state-metrics/0.log" Oct 30 19:47:48.305913 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:48.305872 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qc4zv_79e792e9-28c1-4127-b364-3679fcf27c7c/kube-rbac-proxy-main/0.log" Oct 30 19:47:48.696714 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:48.696675 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-qc4zv_79e792e9-28c1-4127-b364-3679fcf27c7c/kube-rbac-proxy-self/0.log" Oct 30 19:47:49.090127 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:49.090021 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56686866c6-xhcgv_734c247a-5ffb-4ccc-9720-8ba9d5ac3711/metrics-server/0.log" Oct 30 19:47:49.482783 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:49.482752 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-8lpn2_ef44a6cf-b792-4c9c-8503-271d365976f8/monitoring-plugin/0.log" Oct 30 19:47:50.102178 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:50.102154 5869 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ndc8/perf-node-gather-daemonset-69jv5" Oct 30 19:47:52.308328 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:52.308289 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8v99_a1df3f31-27af-4f32-aa1e-e07cf031f2cf/node-exporter/0.log" Oct 30 19:47:52.695506 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:52.695479 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8v99_a1df3f31-27af-4f32-aa1e-e07cf031f2cf/kube-rbac-proxy/0.log" Oct 30 19:47:53.096332 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:53.096252 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8v99_a1df3f31-27af-4f32-aa1e-e07cf031f2cf/init-textfile/0.log" Oct 30 19:47:53.489358 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:53.489328 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hwjnm_7a6523be-9435-46c7-b5a2-b84a2bba0d83/kube-rbac-proxy-main/0.log" Oct 30 19:47:53.879429 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:53.879360 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hwjnm_7a6523be-9435-46c7-b5a2-b84a2bba0d83/kube-rbac-proxy-self/0.log" Oct 30 19:47:54.269962 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:54.269932 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-hwjnm_7a6523be-9435-46c7-b5a2-b84a2bba0d83/openshift-state-metrics/0.log" Oct 30 19:47:54.678825 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:54.678757 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/prometheus/0.log" Oct 30 19:47:55.060523 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:55.060492 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/config-reloader/0.log" Oct 30 19:47:55.452715 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:55.452682 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/thanos-sidecar/0.log" Oct 30 19:47:55.844021 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:55.843939 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/kube-rbac-proxy-web/0.log" Oct 30 19:47:56.235942 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:56.235914 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/kube-rbac-proxy/0.log" Oct 30 19:47:56.626952 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:56.626870 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/kube-rbac-proxy-thanos/0.log" Oct 30 19:47:57.018727 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:57.018700 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bff7919e-2f6b-4c3e-98e4-5195a5c3d2f6/init-config-reloader/0.log" Oct 30 19:47:58.608431 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:58.608403 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bdcd88dbb-94dqj_69ca6d2f-16b4-4af4-be21-0dfec25fdcbb/telemeter-client/0.log" Oct 30 19:47:58.999247 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:58.999214 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bdcd88dbb-94dqj_69ca6d2f-16b4-4af4-be21-0dfec25fdcbb/reload/0.log" Oct 30 19:47:59.280510 ip-10-0-128-38 kubenswrapper[5869]: E1030 19:47:59.280432 5869 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rd622" containerName="extractor" Oct 30 19:47:59.391178 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:59.391146 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bdcd88dbb-94dqj_69ca6d2f-16b4-4af4-be21-0dfec25fdcbb/kube-rbac-proxy/0.log" Oct 30 19:47:59.787864 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:47:59.787836 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/thanos-query/0.log" Oct 30 19:48:00.177542 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:48:00.177460 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/kube-rbac-proxy-web/0.log" Oct 30 19:48:00.567002 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:48:00.566967 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/kube-rbac-proxy/0.log" Oct 30 19:48:00.956304 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:48:00.956266 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/prom-label-proxy/0.log" Oct 30 19:48:01.347435 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:48:01.347343 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/kube-rbac-proxy-rules/0.log" Oct 30 19:48:01.743687 ip-10-0-128-38 kubenswrapper[5869]: I1030 19:48:01.743659 5869 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c8658f985-v8kgg_bdde0beb-f496-4013-aff3-3706e199e7f2/kube-rbac-proxy-metrics/0.log"