Oct 30 18:28:43.637142 ip-10-0-136-105 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 18:28:43.637154 ip-10-0-136-105 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 18:28:43.637161 ip-10-0-136-105 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 18:28:43.637415 ip-10-0-136-105 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 18:28:53.886900 ip-10-0-136-105 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 18:28:53.886917 ip-10-0-136-105 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5caf93184e2a48febe12b40c781dfe64 -- Oct 30 18:30:54.506987 ip-10-0-136-105 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:30:54.915764 ip-10-0-136-105 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:54.915764 ip-10-0-136-105 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:30:54.915764 ip-10-0-136-105 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:54.915764 ip-10-0-136-105 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:54.919347 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919265 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.919347 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919347 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.919347 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919351 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919355 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919359 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919362 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919365 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919368 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919372 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919374 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919377 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919380 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919383 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919386 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919388 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919391 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919393 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919396 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919399 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919401 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919405 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919408 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.919463 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919410 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919413 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919416 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919418 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919421 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919423 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919426 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919429 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919432 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919434 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919437 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919440 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919442 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919446 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919449 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919452 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919454 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919457 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919460 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.919942 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919462 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919465 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919468 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919471 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919473 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919477 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919482 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919486 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919489 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919492 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919494 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919497 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919499 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919502 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919505 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919507 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919510 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919513 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919516 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.920386 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919519 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919521 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919524 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919526 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919529 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919532 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919534 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919538 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919541 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919543 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919546 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919549 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919551 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919554 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919556 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919559 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919562 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919564 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919567 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919569 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.920871 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919571 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919574 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919577 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919580 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919582 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919585 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919587 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919590 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919592 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919595 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919597 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919600 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919603 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919605 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919607 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919611 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919615 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919618 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919621 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.921362 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919624 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919627 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919630 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919632 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919635 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919638 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.919642 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920427 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920436 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920440 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920443 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920446 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920448 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920451 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920454 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920456 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920459 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920462 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920465 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.921818 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920467 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920470 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920473 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920475 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920478 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920481 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920484 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920486 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920488 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920491 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920493 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920496 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920498 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920501 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920505 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920508 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920510 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920513 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920516 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920520 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.922267 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920524 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920528 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920530 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920533 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920536 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920538 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920541 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920543 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920546 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920548 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920551 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920553 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920556 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920558 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920561 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920564 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920567 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920569 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920572 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920574 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.922758 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920577 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920579 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920582 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920585 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920587 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920591 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920594 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920596 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920599 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920602 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920604 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920607 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920609 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920613 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920616 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920618 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920621 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920623 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920625 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920628 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.923237 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920630 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920633 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920636 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920638 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920641 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920648 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920651 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920653 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920656 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920659 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920661 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920664 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920666 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920669 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920672 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920674 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920677 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920679 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920682 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.923697 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920685 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920687 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920690 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920692 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920695 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920698 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920700 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920703 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920720 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920723 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920725 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920728 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920731 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920734 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.920737 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921215 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921219 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921222 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921227 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.924157 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921230 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921233 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921235 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921238 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921241 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921243 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921246 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921249 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921252 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921255 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921257 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921260 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921263 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921266 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921269 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921271 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921274 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921276 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921279 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.924625 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921281 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921284 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921287 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921289 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921292 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921295 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921297 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921300 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921302 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921304 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921307 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921309 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921313 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921316 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921318 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921321 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921323 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921326 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921329 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921331 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.925093 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921334 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921336 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921339 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921341 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921344 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921347 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921349 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921352 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921355 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921357 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921360 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921363 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921368 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921371 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921375 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921378 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921382 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921385 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921387 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.925558 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921390 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921393 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921396 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921400 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921402 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921406 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921409 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921411 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921414 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921416 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921419 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921422 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921424 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921427 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921429 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921432 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921435 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921438 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921440 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.926048 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921443 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921446 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921448 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921450 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921453 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921456 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921458 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921461 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921463 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921466 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921469 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921472 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921474 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921477 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921479 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921482 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921487 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921489 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921493 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921495 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.926499 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921498 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921500 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921504 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921506 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.921508 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921606 2580 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921624 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921632 2580 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921639 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921644 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921647 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921652 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921656 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921659 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921662 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921666 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921669 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921673 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921676 2580 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921679 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921682 2580 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921684 2580 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921688 2580 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:30:54.926983 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921693 2580 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921696 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921699 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921703 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921717 2580 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921726 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921731 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921736 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921739 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921742 2580 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921745 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921749 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921753 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921756 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921760 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921763 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921766 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921769 2580 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921772 2580 flags.go:64] FLAG: --enable-server="true" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921775 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921780 2580 flags.go:64] FLAG: --event-burst="100" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921783 2580 flags.go:64] FLAG: --event-qps="50" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921786 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921789 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:30:54.927535 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921792 2580 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921797 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921800 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921803 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921806 2580 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921809 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921812 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921815 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921819 2580 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921822 2580 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921825 2580 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921828 2580 flags.go:64] FLAG: --feature-gates="" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921833 2580 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921837 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921840 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921845 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921848 2580 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921851 2580 flags.go:64] FLAG: --help="false" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921854 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-105.ec2.internal" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921857 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921861 2580 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921864 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921867 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:30:54.928140 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921870 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921874 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921877 2580 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921879 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921882 2580 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921885 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921889 2580 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921892 2580 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921895 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921897 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921901 2580 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921904 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921906 2580 flags.go:64] FLAG: --lock-file="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921909 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921912 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921916 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921921 2580 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921924 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921928 2580 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921931 2580 flags.go:64] FLAG: --logging-format="text" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921934 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921937 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921942 2580 flags.go:64] FLAG: --manifest-url="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921944 2580 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921950 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:30:54.928803 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921953 2580 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921957 2580 flags.go:64] FLAG: --max-pods="110" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921961 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921964 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921967 2580 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921970 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921973 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921976 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921978 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921986 2580 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921989 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921992 2580 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921996 2580 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.921998 2580 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922001 2580 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922004 2580 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922007 2580 flags.go:64] FLAG: --port="10250" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922010 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922013 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0173b6aac6f7db1b0" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922016 2580 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922019 2580 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922022 2580 flags.go:64] FLAG: --register-node="true" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922025 2580 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922029 2580 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922032 2580 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922035 2580 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:30:54.929868 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922038 2580 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922043 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922046 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922050 2580 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922053 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922057 2580 flags.go:64] FLAG: --runonce="false" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922060 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922063 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922067 2580 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922070 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922073 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922076 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922079 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922082 2580 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922085 2580 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922088 2580 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922091 2580 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922094 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922097 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922101 2580 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922104 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922109 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922112 2580 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922115 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922123 2580 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:30:54.930962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922126 2580 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922129 2580 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922132 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922135 2580 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922138 2580 flags.go:64] FLAG: --v="2" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922147 2580 flags.go:64] FLAG: --version="false" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922151 2580 flags.go:64] FLAG: --vmodule="" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922155 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922159 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922293 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922297 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922300 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922304 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922307 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922310 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922313 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922315 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922318 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922322 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922326 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922329 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.931685 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922332 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922335 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922337 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922340 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922343 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922345 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922348 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922351 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922353 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922356 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922358 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922361 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922363 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922366 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922377 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922380 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922382 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922385 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922388 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922390 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.932346 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922394 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922397 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922399 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922402 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922405 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922407 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922410 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922412 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922415 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922418 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922420 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922423 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922425 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922428 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922431 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922433 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922436 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922438 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922441 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922443 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.932891 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922446 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922448 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922451 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922453 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922456 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922458 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922461 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922464 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922466 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922469 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922471 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922474 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922476 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922479 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922482 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922485 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922487 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922490 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922493 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.933728 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922495 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922498 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922500 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922503 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922505 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922508 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922510 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922513 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922515 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922518 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922521 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922523 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922526 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922528 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922531 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922533 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922536 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922538 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922541 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922543 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.934448 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922546 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922548 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922551 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922553 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922556 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922558 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922561 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922564 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922567 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922569 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922572 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922574 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922578 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922582 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922585 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.922613 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:54.935208 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922647 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922651 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922654 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922657 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922659 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922662 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922665 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922668 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922670 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922673 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922675 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922678 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922681 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922683 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922686 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922688 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922690 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922693 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922696 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922698 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.935896 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922700 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922703 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922719 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922721 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922726 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922729 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922731 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922734 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922737 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922739 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922742 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922744 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922747 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922750 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922752 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922755 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922757 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922760 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922763 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922766 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.936389 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922769 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922771 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922774 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922777 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922779 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922782 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922784 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922787 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922790 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922793 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922795 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922798 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922800 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922803 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922807 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922810 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922814 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922817 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922820 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.936907 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922822 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922825 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922828 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922830 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922833 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922835 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922838 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922840 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922843 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922846 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922849 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922851 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922854 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922856 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922859 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922862 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922864 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922867 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922869 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922872 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.937366 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922874 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922877 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922879 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922882 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922884 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922887 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922889 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922893 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922896 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922899 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922903 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922907 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922910 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922913 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922916 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922919 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922921 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922924 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922926 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.937844 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922929 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922931 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922934 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922938 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922942 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922945 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922948 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.922952 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.931108 2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.931128 2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.932771 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.932786 2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932883 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932893 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932899 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932904 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932909 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932914 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.938284 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932918 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932923 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932927 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932932 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932936 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932941 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932945 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932950 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932954 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932958 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932962 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932967 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932972 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932976 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932980 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932984 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932989 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932993 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.932998 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.938756 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933002 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933007 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933011 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933016 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933023 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933028 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933032 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933037 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933041 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933045 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933049 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933054 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933058 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933062 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933069 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933076 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933082 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933088 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933094 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.939192 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933099 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933103 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933107 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933112 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933116 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933120 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933125 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933128 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933133 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933138 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933142 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933146 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933150 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933154 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933159 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933163 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933167 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933173 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933178 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933183 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.939636 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933188 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933192 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933196 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933200 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933205 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933209 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933213 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933217 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933221 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933227 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933231 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933236 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933240 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933244 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933248 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933252 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933257 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933261 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933265 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933269 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.940118 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933274 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933278 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933282 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933286 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933290 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933294 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933299 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933303 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933308 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933313 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933317 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933322 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933327 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933331 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933335 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933339 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933344 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933349 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933353 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933357 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.940572 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933361 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.933365 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.933408 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.935978 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.935995 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936001 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936006 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936010 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936015 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936020 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936026 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936030 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936034 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936038 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936042 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936047 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.941077 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936051 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936055 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936060 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936064 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936068 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936073 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936077 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936081 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936085 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936090 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936094 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936099 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936103 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936108 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936113 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936118 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936122 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936127 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936131 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936136 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.941504 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936140 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936145 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936149 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936154 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936158 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936163 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936166 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936169 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936171 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936174 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936177 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936179 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936182 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936186 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936191 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936194 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936197 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936200 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936202 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.941985 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936205 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936208 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936211 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936213 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936217 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936221 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936223 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936226 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936229 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936232 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936234 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936237 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936240 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936243 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936246 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936249 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936252 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936254 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936257 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.942459 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936260 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936262 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936265 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936268 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936272 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936275 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936278 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936281 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936284 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936287 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936289 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936292 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936296 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936299 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936301 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936304 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936306 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936309 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936312 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936314 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.942924 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936317 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936319 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936322 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936325 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936327 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936330 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936333 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936335 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936338 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936341 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936343 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936346 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936348 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936351 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936353 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.936385 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:54.943387 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936424 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936429 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936432 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936436 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936439 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936443 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936447 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936450 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936454 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936457 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936460 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936462 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936465 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936469 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936471 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936474 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936477 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936479 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936482 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:54.943770 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936485 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936487 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936490 2580 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936494 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936498 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936500 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936503 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936506 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936509 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936511 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936514 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936517 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936519 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936522 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936525 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936527 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936530 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936532 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936535 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:54.944216 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936538 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936540 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936543 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936545 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936548 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936551 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936554 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936557 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936560 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936562 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936565 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936568 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936570 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936573 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936575 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936578 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936582 2580 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936585 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936587 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936590 2580 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:54.944678 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936593 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936596 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936598 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936601 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936604 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936606 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936609 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936612 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936614 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936617 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936619 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936622 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936626 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936628 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936631 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936633 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936636 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936639 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936642 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936645 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:54.945235 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936648 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936650 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936653 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936655 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936659 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936662 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936665 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936668 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936672 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936675 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936677 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936680 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936683 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936686 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936688 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936691 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936694 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936696 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:54.945689 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936699 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936702 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936759 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936763 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936766 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936769 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936772 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936774 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936777 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:54.936781 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.936849 2580 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.941090 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.942222 2580 server.go:1014] "Starting client certificate rotation" Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.942315 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.943198 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:54.946127 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.944536 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:54.946557 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.946177 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:54.958652 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.958632 2580 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:30:54.963290 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.963274 2580 log.go:25] "Validated CRI v1 image API" Oct 30 18:30:54.964152 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.964126 2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:30:54.966914 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.966894 2580 fs.go:135] Filesystem UUIDs: map[1f37733c-6da4-4440-beb6-301689f22fd0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e68c8ab5-5873-4257-acd4-114be8244df8:/dev/nvme0n1p3] Oct 30 18:30:54.967007 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.966914 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:30:54.973550 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.973421 2580 manager.go:217] Machine: {Timestamp:2026-10-30 18:30:54.971982169 +0000 UTC m=+0.371879528 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100171 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239e5d8c30c91a2b2058514dcb09a4 SystemUUID:ec239e5d-8c30-c91a-2b20-58514dcb09a4 BootID:5caf9318-4e2a-48fe-be12-b40c781dfe64 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d1:62:cf:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d1:62:cf:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:01:a1:63:4c:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:30:54.973550 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.973542 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:30:54.973724 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.973649 2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:30:54.977074 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977054 2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:30:54.977204 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977075 2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:30:54.977252 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977205 2580 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:30:54.977252 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977214 2580 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:30:54.977306 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977254 2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:54.977306 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.977271 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:54.978530 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.978520 2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:30:54.983190 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.983178 2580 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:30:54.983304 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.983295 2580 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:30:54.984109 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984099 2580 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:30:54.984141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984113 2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:30:54.984141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984124 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:30:54.984141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984133 2580 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:30:54.984141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984141 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:30:54.984541 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.984521 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:54.988037 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.988014 2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:30:54.995560 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.995544 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:54.996390 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996379 2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:30:54.996432 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996408 2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:30:54.996555 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996545 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:30:54.996585 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996565 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:30:54.996585 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996577 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:30:54.996585 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996585 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:30:54.996662 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996591 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:30:54.996662 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.996597 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:30:54.997303 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997295 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:30:54.997340 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997304 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:30:54.997340 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997311 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:30:54.997340 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997318 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:30:54.997340 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997326 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:30:54.997340 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.997335 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:30:54.998069 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.998060 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:30:54.998069 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:54.998069 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:30:55.001997 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.001983 2580 server.go:1263] "Started kubelet" Oct 30 18:30:55.002123 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.002089 2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:30:55.002123 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.002083 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:30:55.002258 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.002159 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:30:55.002866 ip-10-0-136-105 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:30:55.003503 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.003485 2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:30:55.004333 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.004313 2580 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:30:55.009256 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.009238 2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:30:55.012418 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.012403 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:30:55.013043 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.013017 2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:30:55.013136 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.013118 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.013995 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.013971 2580 factory.go:55] Registering systemd factory Oct 30 18:30:55.013995 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.013991 2580 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:30:55.014209 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014124 2580 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:30:55.014209 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014124 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:30:55.014209 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014151 2580 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:30:55.014330 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014244 2580 factory.go:153] Registering CRI-O factory Oct 30 18:30:55.014330 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014256 2580 factory.go:223] Registration of the crio container factory successfully Oct 30 18:30:55.014330 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014305 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:30:55.014330 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014319 2580 factory.go:103] Registering Raw factory Oct 30 18:30:55.014330 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014255 2580 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:30:55.014511 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014342 2580 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:30:55.014511 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.014330 2580 manager.go:1199] Started watching for new ooms in manager Oct 30 18:30:55.015492 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.015476 2580 manager.go:319] Starting recovery of all containers Oct 30 18:30:55.027366 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.027346 2580 manager.go:324] Recovery completed Oct 30 18:30:55.032802 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.032779 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:30:55.033142 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.033128 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.033965 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.032779 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-105.ec2.internal.18e36234ace2ab2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-105.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-105.ec2.internal,},FirstTimestamp:2026-10-30 18:30:55.001946925 +0000 UTC m=+0.401844284,LastTimestamp:2026-10-30 18:30:55.001946925 +0000 UTC m=+0.401844284,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-105.ec2.internal,}" Oct 30 18:30:55.034962 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.034945 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.035024 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.034974 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.035024 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.034988 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.035559 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.035544 2580 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:30:55.035559 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.035556 2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:30:55.035559 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.035560 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.035697 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.035573 2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:30:55.036224 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.036205 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.036294 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.036240 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.036294 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.036254 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.037833 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.037821 2580 policy_none.go:50] "Start" Oct 30 18:30:55.037873 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.037837 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:30:55.037873 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.037851 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:30:55.039864 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.039853 2580 policy_none.go:44] "Start" Oct 30 18:30:55.042261 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.042237 2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:30:55.047097 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.047080 2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:30:55.055543 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.055520 2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 18:30:55.055627 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.055498 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-105.ec2.internal.18e36234aeda6759 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-105.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-105.ec2.internal,},FirstTimestamp:2026-10-30 18:30:55.034959705 +0000 UTC m=+0.434857063,LastTimestamp:2026-10-30 18:30:55.034959705 +0000 UTC m=+0.434857063,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-105.ec2.internal,}" Oct 30 18:30:55.055808 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.055790 2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:30:55.055861 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.055853 2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 18:30:55.060321 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.060305 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s85hl" Oct 30 18:30:55.072639 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.072619 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s85hl" Oct 30 18:30:55.076717 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.076638 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-105.ec2.internal.18e36234aedaba5e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-105.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-105.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-105.ec2.internal,},FirstTimestamp:2026-10-30 18:30:55.034980958 +0000 UTC m=+0.434878316,LastTimestamp:2026-10-30 18:30:55.034980958 +0000 UTC m=+0.434878316,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-105.ec2.internal,}" Oct 30 18:30:55.080605 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.080399 2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:30:55.084636 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.084621 2580 manager.go:341] "Starting Device Plugin manager" Oct 30 18:30:55.084721 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.084653 2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:30:55.084721 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.084665 2580 server.go:87] "Starting device plugin registration server" Oct 30 18:30:55.084966 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.084954 2580 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:30:55.085004 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.084970 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:30:55.085144 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.085129 2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:30:55.085225 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.085215 2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:30:55.085225 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.085225 2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:30:55.085793 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.085769 2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:30:55.085896 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.085812 2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.186031 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.185965 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.188075 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.186913 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.188075 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.186941 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.188075 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.186951 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.188075 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.186981 2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.190957 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.190930 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:30:55.192058 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.192039 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:30:55.192058 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.192057 2580 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:30:55.192202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.192088 2580 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:30:55.192202 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.192119 2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 18:30:55.207251 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.207228 2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.207371 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.207265 2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-105.ec2.internal\": node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.209012 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.208994 2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:55.249210 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.249180 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.292404 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.292372 2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal"] Oct 30 18:30:55.292480 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.292441 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.293433 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.293415 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.293550 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.293443 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.293550 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.293457 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.295773 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.295758 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.295933 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.295915 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.296000 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.295955 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.296461 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296442 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.296539 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296474 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.296539 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296484 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.296539 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296446 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.296628 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296542 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.296628 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.296555 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.298649 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.298634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.298735 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.298660 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:55.299320 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.299305 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:55.299384 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.299333 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:55.299384 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.299342 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:55.316027 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.316004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.316128 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.316030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.316128 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.316045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.325449 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.325427 2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-105.ec2.internal\" not found" node="ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.328263 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.328248 2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-105.ec2.internal\" not found" node="ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.349986 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.349962 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.416333 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.416333 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.416463 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.416463 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.416463 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416396 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.416463 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.416423 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.450472 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.450407 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.551004 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.550979 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.629103 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.629063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.631222 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.631202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:55.651185 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.651163 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.751831 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.751746 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.852318 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.852271 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:55.941928 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.941893 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:30:55.942388 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:55.942030 2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 18:30:55.953197 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:55.953174 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:57.309818 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:57.309744 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:57.334856 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.334817 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:56 +0000 UTC" deadline="2028-05-21 05:54:46.662217135 +0000 UTC" Oct 30 18:30:57.335029 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.334866 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13643h23m49.327363284s" Oct 30 18:30:57.410467 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:57.410428 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:57.430486 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:57.430445 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a961c4fb7c61fa12bec5d71405aa03.slice/crio-c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89 WatchSource:0}: Error finding container c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89: Status 404 returned error can't find the container with id c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89 Oct 30 18:30:57.439565 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.439530 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:30:57.440247 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.440216 2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:57.449563 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:57.449529 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af086efb620e723a004b0f8b9bdafb8.slice/crio-fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978 WatchSource:0}: Error finding container fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978: Status 404 returned error can't find the container with id fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978 Oct 30 18:30:57.451473 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.451438 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" event={"ID":"c2a961c4fb7c61fa12bec5d71405aa03","Type":"ContainerStarted","Data":"c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89"} Oct 30 18:30:57.485826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.485805 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:57.487082 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.487064 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:57.487150 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.487096 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:57.487150 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.487105 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:57.509603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.509578 2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:57.510596 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:57.510578 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:57.611193 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:57.611113 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:30:57.646233 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.646212 2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:57.669242 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.669219 2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:30:57.693453 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.693426 2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:30:57.694205 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.694185 2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:30:57.718495 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:57.718469 2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:30:58.240888 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.240857 2580 apiserver.go:52] "Watching apiserver" Oct 30 18:30:58.269907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.269879 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:30:58.287636 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.287609 2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:30:58.288018 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.287997 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-fx675","openshift-multus/multus-additional-cni-plugins-bpfcd","openshift-multus/network-metrics-daemon-km5cq","openshift-network-diagnostics/network-check-target-x7rq7","openshift-network-operator/iptables-alerter-5xlpr","kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal","kube-system/konnectivity-agent-rl74h","openshift-ovn-kubernetes/ovnkube-node-6v4xj","kube-system/global-pull-secret-syncer-t6kkd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44","openshift-cluster-node-tuning-operator/tuned-mlq92","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal","openshift-multus/multus-vsk5n","openshift-dns/node-resolver-9r555"] Oct 30 18:30:58.290651 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.290629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.293681 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.293659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.295036 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.295013 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.295138 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.295088 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:30:58.297278 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.297262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:30:58.297358 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.297310 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:30:58.299300 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.299280 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.301523 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.301502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.303998 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.303982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.304079 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.304044 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:30:58.306003 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.305982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.307409 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.307373 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:30:58.307516 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.307498 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:30:58.307637 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.307577 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:30:58.308322 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.308304 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.310044 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.310023 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.311119 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.311101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.316017 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.315990 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:30:58.316098 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.316088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.317107 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.316305 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.317107 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.316856 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:30:58.317107 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.317000 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:30:58.318033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.318011 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.329561 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.329346 2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:30:58.329561 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.329412 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xrhdc" Oct 30 18:30:58.372160 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.372126 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:30:58.382460 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.382603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.382603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.382603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.382603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382644 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.382738 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382722 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.382907 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382878 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78st7\" (UniqueName: \"kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.382993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbdj5\" (UniqueName: \"kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.383170 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:30:58.383521 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzc4f\" (UniqueName: \"kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnz9\" (UniqueName: \"kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcmw8\" (UniqueName: \"kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.383853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.383982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjb5\" (UniqueName: \"kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.384250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.384631 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66z2r\" (UniqueName: \"kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384775 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj4vw\" (UniqueName: \"kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384891 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.385023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.384992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9z8f\" (UniqueName: \"kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.385577 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.385219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.409609 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409578 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:30:58.409799 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409778 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:30:58.409858 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409802 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.409909 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409855 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.409909 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409888 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:30:58.409991 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.409935 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:30:58.410023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410005 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:30:58.410023 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410017 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:30:58.410470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410314 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.410470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410427 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:30:58.410470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410436 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:30:58.410470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410429 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.410470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410455 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.410773 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410526 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.410773 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.410601 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411026 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411058 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411077 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411097 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411106 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:30:58.411181 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.411152 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.413128 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.413111 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.413128 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.413123 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:30:58.413128 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.413112 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:30:58.413297 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.413200 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:30:58.424280 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.424262 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:30:58.424388 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.424321 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:30:58.424442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.424389 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:30:58.453905 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.453871 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerStarted","Data":"fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978"} Oct 30 18:30:58.486396 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.486583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.486583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.486583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.486583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzc4f\" (UniqueName: \"kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.486756 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnz9\" (UniqueName: \"kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcmw8\" (UniqueName: \"kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.486987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjb5\" (UniqueName: \"kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66z2r\" (UniqueName: \"kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487651 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.487874 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.488098 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.487910 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.488424 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488391 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.488424 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488391 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.488629 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488598 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.488741 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488672 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.488741 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488718 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.488835 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.488835 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.488820 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.489033 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.489115 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489034 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.489115 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.489115 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489093 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.489252 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.489252 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.489421 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.489934 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489910 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.490017 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.489957 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.490131 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.490426 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.490509 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490413 2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:30:58.490509 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490471 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.490838 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490810 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.490928 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.490884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491026 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.491181 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491298 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.491442 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.491524 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:58.991496566 +0000 UTC m=+3.135562299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491561 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491641 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj4vw\" (UniqueName: \"kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491701 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.491992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491858 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.491986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9z8f\" (UniqueName: \"kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492086 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.492826 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492363 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492423 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492642 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78st7\" (UniqueName: \"kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492710 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.493456 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.492458 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbdj5\" (UniqueName: \"kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493833 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.494146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.493900 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.494458 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.494217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.494458 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.494370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.494458 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.494444 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.494804 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.494779 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.494922 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.494899 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.495360 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.496085 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.496148 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:30:58.996123289 +0000 UTC m=+3.140189042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.496795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497481 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497579 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497598 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497678 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.497945 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.498133 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.499202 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.498375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.500074 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.499576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.518382 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.518356 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzc4f\" (UniqueName: \"kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.519967 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.519941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcmw8\" (UniqueName: \"kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.526109 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.526082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj4vw\" (UniqueName: \"kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.526226 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.526123 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnz9\" (UniqueName: \"kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.526462 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.526438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjb5\" (UniqueName: \"kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.526550 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.526445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66z2r\" (UniqueName: \"kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.527478 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.527447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78st7\" (UniqueName: \"kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:58.529067 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.529048 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9z8f\" (UniqueName: \"kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.530275 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.530253 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:58.530363 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.530278 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:58.530363 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.530292 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:58.530363 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:58.530352 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:30:59.030331671 +0000 UTC m=+3.174397417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:58.532650 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.532619 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbdj5\" (UniqueName: \"kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.586851 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.586817 2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:58.617616 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.617581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:30:58.621361 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.621330 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vsk5n" Oct 30 18:30:58.631272 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.631253 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:30:58.631695 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.631667 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5fa788_50a7_47bb_bfb3_231034995cea.slice/crio-f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e WatchSource:0}: Error finding container f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e: Status 404 returned error can't find the container with id f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e Oct 30 18:30:58.632418 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.632398 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b81f81_6c7f_44cd_a040_e7488cfea341.slice/crio-6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f WatchSource:0}: Error finding container 6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f: Status 404 returned error can't find the container with id 6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f Oct 30 18:30:58.637716 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.637692 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9r555" Oct 30 18:30:58.643247 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.643222 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346d01a4_c662_4cb1_bdd7_a2e39ee5c6c3.slice/crio-5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3 WatchSource:0}: Error finding container 5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3: Status 404 returned error can't find the container with id 5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3 Oct 30 18:30:58.644193 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.644163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fx675" Oct 30 18:30:58.648826 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.648802 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode025fb6d_2541_4243_a8d3_8fc4fc5f5940.slice/crio-eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2 WatchSource:0}: Error finding container eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2: Status 404 returned error can't find the container with id eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2 Oct 30 18:30:58.654659 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.654631 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7a8c1d_004b_423c_8d4c_27e2b6757aae.slice/crio-3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2 WatchSource:0}: Error finding container 3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2: Status 404 returned error can't find the container with id 3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2 Oct 30 18:30:58.667586 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.667524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:30:58.669653 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.669629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:30:58.674686 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.674634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:30:58.680198 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.680170 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b05773_655d_4d2e_a924_2a6746e2a1e6.slice/crio-670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f WatchSource:0}: Error finding container 670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f: Status 404 returned error can't find the container with id 670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f Oct 30 18:30:58.681076 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.681041 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cc86a6_a954_4a74_a70b_63e31860233c.slice/crio-7b5c2158b1d119854ad5e97198c3b3e5579fafaec51ce2f1255935c9949accd1 WatchSource:0}: Error finding container 7b5c2158b1d119854ad5e97198c3b3e5579fafaec51ce2f1255935c9949accd1: Status 404 returned error can't find the container with id 7b5c2158b1d119854ad5e97198c3b3e5579fafaec51ce2f1255935c9949accd1 Oct 30 18:30:58.686376 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.686348 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3423a0_15ee_4a92_a819_0507cf101d2c.slice/crio-9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe WatchSource:0}: Error finding container 9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe: Status 404 returned error can't find the container with id 9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe Oct 30 18:30:58.689941 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:58.689921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:30:58.702449 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:30:58.702420 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbee4f62_3043_46ce_821a_b77f3746f83a.slice/crio-114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b WatchSource:0}: Error finding container 114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b: Status 404 returned error can't find the container with id 114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b Oct 30 18:30:59.001283 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.001244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:30:59.001433 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.001308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:30:59.001433 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.001385 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:59.001433 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.001386 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:59.001433 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.001434 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.00142234 +0000 UTC m=+4.145488069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:59.001672 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.001447 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.001441017 +0000 UTC m=+4.145506746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:59.103217 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.102361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:30:59.103217 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.102803 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:59.103217 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.102821 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:59.103217 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.102831 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:59.103217 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:30:59.102901 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.102887687 +0000 UTC m=+4.246953429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:59.332630 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.332490 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:58 +0000 UTC" deadline="2028-05-07 08:20:56.089813763 +0000 UTC" Oct 30 18:30:59.332630 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.332567 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13309h49m56.757251578s" Oct 30 18:30:59.472194 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.472156 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rl74h" event={"ID":"ba5fa788-50a7-47bb-bfb3-231034995cea","Type":"ContainerStarted","Data":"f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e"} Oct 30 18:30:59.479971 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.479933 2580 generic.go:356] "Generic (PLEG): container finished" podID="8af086efb620e723a004b0f8b9bdafb8" containerID="49c728f2745051b6f69f8131952a098d260ade953080b68807a7c710388c90a5" exitCode=0 Oct 30 18:30:59.480188 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.480026 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerDied","Data":"49c728f2745051b6f69f8131952a098d260ade953080b68807a7c710388c90a5"} Oct 30 18:30:59.497922 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.497882 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b"} Oct 30 18:30:59.513149 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.513081 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" event={"ID":"1f3423a0-15ee-4a92-a819-0507cf101d2c","Type":"ContainerStarted","Data":"9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe"} Oct 30 18:30:59.518566 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.518458 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mlq92" event={"ID":"24cc86a6-a954-4a74-a70b-63e31860233c","Type":"ContainerStarted","Data":"7b5c2158b1d119854ad5e97198c3b3e5579fafaec51ce2f1255935c9949accd1"} Oct 30 18:30:59.524509 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.524440 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9r555" event={"ID":"e025fb6d-2541-4243-a8d3-8fc4fc5f5940","Type":"ContainerStarted","Data":"eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2"} Oct 30 18:30:59.549229 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.548505 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" event={"ID":"c2a961c4fb7c61fa12bec5d71405aa03","Type":"ContainerStarted","Data":"7b273e1bbb5291102816a3fb98b6f3fb375578f274e9281b1e3aa2021a424c89"} Oct 30 18:30:59.567639 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.567560 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerStarted","Data":"670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f"} Oct 30 18:30:59.575922 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.575849 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fx675" event={"ID":"cf7a8c1d-004b-423c-8d4c-27e2b6757aae","Type":"ContainerStarted","Data":"3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2"} Oct 30 18:30:59.589573 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.589431 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5xlpr" event={"ID":"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3","Type":"ContainerStarted","Data":"5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3"} Oct 30 18:30:59.591633 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.591581 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" podStartSLOduration=2.59156429 podStartE2EDuration="2.59156429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:59.590702491 +0000 UTC m=+3.734768240" watchObservedRunningTime="2026-10-30 18:30:59.59156429 +0000 UTC m=+3.735630040" Oct 30 18:30:59.597489 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:30:59.597461 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vsk5n" event={"ID":"a3b81f81-6c7f-44cd-a040-e7488cfea341","Type":"ContainerStarted","Data":"6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f"} Oct 30 18:31:00.008615 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.008468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:00.008615 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.008571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:00.008832 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.008707 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.008832 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.008772 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:02.008752542 +0000 UTC m=+6.152818270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.009211 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.009090 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.009211 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.009161 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:02.009144926 +0000 UTC m=+6.153210666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.109285 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.109248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:00.109451 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.109432 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:00.109451 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.109451 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:00.109595 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.109464 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:00.109595 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.109520 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:31:02.109502594 +0000 UTC m=+6.253568340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:00.332941 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.332851 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:58 +0000 UTC" deadline="2028-06-25 12:23:43.694995518 +0000 UTC" Oct 30 18:31:00.332941 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.332892 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14489h52m43.362107276s" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.451798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.451919 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.451998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.452092 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.452156 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:00.452651 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:00.452239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:00.610957 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:00.610785 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerStarted","Data":"41e3cd50b32e21b93774c19413646bdf190f9714b3937103ff1c6406969ed37c"} Oct 30 18:31:01.618812 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:01.618690 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerStarted","Data":"fd1828088bf9e02ea21f391dfde3a5e5976a411648a914e8c09aec143e0679b8"} Oct 30 18:31:01.691901 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:01.691838 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" podStartSLOduration=4.691819877 podStartE2EDuration="4.691819877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:00.65966816 +0000 UTC m=+4.803733913" watchObservedRunningTime="2026-10-30 18:31:01.691819877 +0000 UTC m=+5.835885629" Oct 30 18:31:02.024112 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.024027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:02.024276 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.024150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:02.024344 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.024288 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:02.024396 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.024361 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:06.024337651 +0000 UTC m=+10.168403419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:02.024824 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.024804 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:02.024902 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.024873 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:06.024857962 +0000 UTC m=+10.168923709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:02.125335 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.124676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:02.125335 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.124885 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:02.125335 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.124905 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:02.125335 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.124919 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:02.125335 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.124979 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:31:06.124958023 +0000 UTC m=+10.269023771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.451058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.451189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.451593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.451692 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.451761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:02.451911 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:02.451844 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:02.631114 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.630137 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="fd1828088bf9e02ea21f391dfde3a5e5976a411648a914e8c09aec143e0679b8" exitCode=0 Oct 30 18:31:02.631114 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.630243 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"fd1828088bf9e02ea21f391dfde3a5e5976a411648a914e8c09aec143e0679b8"} Oct 30 18:31:02.644999 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.644963 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rl74h" event={"ID":"ba5fa788-50a7-47bb-bfb3-231034995cea","Type":"ContainerStarted","Data":"377d8811aa62dbd6b4b63dae268041c908e789c94a19643f8eb880b88da99868"} Oct 30 18:31:02.747992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:02.747943 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rl74h" podStartSLOduration=3.003768895 podStartE2EDuration="6.747929338s" totalImagesPullingTime="3.744160443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:02.747013953 +0000 UTC m=+6.891079705" watchObservedRunningTime="2026-10-30 18:31:02.747929338 +0000 UTC m=+6.891995089" Oct 30 18:31:03.655056 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:03.654977 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fx675" event={"ID":"cf7a8c1d-004b-423c-8d4c-27e2b6757aae","Type":"ContainerStarted","Data":"af7b53f09083c80743a6097a94b288a222caf5f1cba4fe288cc043ebca4d02f9"} Oct 30 18:31:04.450572 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:04.450073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:04.450572 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:04.450213 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:04.451458 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:04.451295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:04.451458 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:04.451397 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:04.453000 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:04.452829 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:04.453000 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:04.452953 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:04.659695 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:04.659616 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" event={"ID":"1f3423a0-15ee-4a92-a819-0507cf101d2c","Type":"ContainerStarted","Data":"e5caa9d74aa07227f749e52f9c2e894f73fbb8789f6e22fc06ae4a0f83ebc2de"} Oct 30 18:31:05.991388 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:05.991339 2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.061608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.061713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.061864 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.061931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:14.061911406 +0000 UTC m=+18.205977158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.062001 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:06.062475 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.062036 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:14.062024747 +0000 UTC m=+18.206090493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:06.162651 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.162609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:06.162815 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.162802 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:06.162895 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.162826 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:06.162895 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.162841 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:06.163765 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.162902 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:31:14.162882126 +0000 UTC m=+18.306947878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:06.346323 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.345931 2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:31:05.991354873Z","UUID":"a455a11e-da53-4dbb-9f5c-0eafb9e72bb5","Handler":null,"Name":"","Endpoint":""} Oct 30 18:31:06.348459 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.347712 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:31:06.348459 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.347740 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.450186 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.450191 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.450275 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.450354 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.450373 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:06.450581 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:06.450465 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:06.670246 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.670209 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" event={"ID":"1f3423a0-15ee-4a92-a819-0507cf101d2c","Type":"ContainerStarted","Data":"7ba4cd2cf0fede0c880b116571876093f7f2d1db8254d78e8bf899944d4d60dc"} Oct 30 18:31:06.673344 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.673206 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vsk5n" event={"ID":"a3b81f81-6c7f-44cd-a040-e7488cfea341","Type":"ContainerStarted","Data":"678f8be48ac882866aec74fbc85ac618c6222b35bbc263401f09234c19827a06"} Oct 30 18:31:06.719288 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.719243 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fx675" podStartSLOduration=6.039605477 podStartE2EDuration="10.719225537s" totalImagesPullingTime="4.67962006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:03.701492661 +0000 UTC m=+7.845558436" watchObservedRunningTime="2026-10-30 18:31:06.719225537 +0000 UTC m=+10.863291286" Oct 30 18:31:06.720297 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.720184 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-vsk5n" podStartSLOduration=3.5053646069999997 podStartE2EDuration="10.720170966s" totalImagesPullingTime="7.214806359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:06.719327034 +0000 UTC m=+10.863392786" watchObservedRunningTime="2026-10-30 18:31:06.720170966 +0000 UTC m=+10.864236717" Oct 30 18:31:06.902583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.902303 2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:31:06.903112 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:06.902918 2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:31:07.677565 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.677288 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9r555" event={"ID":"e025fb6d-2541-4243-a8d3-8fc4fc5f5940","Type":"ContainerStarted","Data":"8664a76a05f881dc2ea79b4fba9a81937062f5003f2f3dbf6387476a29b889cf"} Oct 30 18:31:07.679789 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.679764 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="19d9d7af7f4ce6a231d1f9f629c88ec5ae183b665415333a96641dfe33f5fff8" exitCode=0 Oct 30 18:31:07.679919 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.679866 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"19d9d7af7f4ce6a231d1f9f629c88ec5ae183b665415333a96641dfe33f5fff8"} Oct 30 18:31:07.682187 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.682160 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" event={"ID":"1f3423a0-15ee-4a92-a819-0507cf101d2c","Type":"ContainerStarted","Data":"d051837e641294d6260eaddfe2bd7b25cb21fcc371cb59d257602894c3f59168"} Oct 30 18:31:07.739529 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.739474 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-9r555" podStartSLOduration=3.885956898 podStartE2EDuration="11.739455511s" totalImagesPullingTime="7.853498613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:07.738672132 +0000 UTC m=+11.882737884" watchObservedRunningTime="2026-10-30 18:31:07.739455511 +0000 UTC m=+11.883521284" Oct 30 18:31:07.785513 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:07.785460 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" podStartSLOduration=3.803760884 podStartE2EDuration="11.785446291s" totalImagesPullingTime="7.981685407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:07.785092813 +0000 UTC m=+11.929158566" watchObservedRunningTime="2026-10-30 18:31:07.785446291 +0000 UTC m=+11.929512043" Oct 30 18:31:08.448348 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.448312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:08.448348 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.448334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:08.448591 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:08.448461 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:08.448591 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.448518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:08.448688 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:08.448630 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:08.448753 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:08.448730 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:08.686901 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.686812 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mlq92" event={"ID":"24cc86a6-a954-4a74-a70b-63e31860233c","Type":"ContainerStarted","Data":"901d56ebaf41f6d24caab4ecd0ca7df3b6225c5941dd229373d23660812f98ff"} Oct 30 18:31:08.689583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.689551 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5xlpr" event={"ID":"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3","Type":"ContainerStarted","Data":"7fe23748ca410722db7ce6ff500aa76c2e194487c49657e74798ac19f39a073a"} Oct 30 18:31:08.743146 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.743100 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mlq92" podStartSLOduration=3.21750892 podStartE2EDuration="12.743083712s" totalImagesPullingTime="9.525574792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:08.742648408 +0000 UTC m=+12.886714158" watchObservedRunningTime="2026-10-30 18:31:08.743083712 +0000 UTC m=+12.887149466" Oct 30 18:31:08.786423 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:08.786367 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5xlpr" podStartSLOduration=5.19119707 podStartE2EDuration="12.786350708s" totalImagesPullingTime="7.595153638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:08.785815912 +0000 UTC m=+12.929881679" watchObservedRunningTime="2026-10-30 18:31:08.786350708 +0000 UTC m=+12.930416463" Oct 30 18:31:09.547457 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:09.547423 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:31:09.547645 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:09.547618 2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:09.548345 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:09.548319 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:31:09.693401 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:09.693367 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="d50e7886aa01de211502966cf9ac9d2ccf29d411376ecdcd08f29ed36319addf" exitCode=0 Oct 30 18:31:09.693892 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:09.693477 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"d50e7886aa01de211502966cf9ac9d2ccf29d411376ecdcd08f29ed36319addf"} Oct 30 18:31:10.448935 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:10.448893 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:10.449120 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:10.449014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:10.449120 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:10.449026 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:10.449250 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:10.449124 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:10.449250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:10.449184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:10.449348 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:10.449257 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:10.698713 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:10.698684 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerStarted","Data":"499d97ae9783f008d05ee7812083b91d36c9aa86475da55437cae355477e668d"} Oct 30 18:31:11.703741 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:11.703704 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="499d97ae9783f008d05ee7812083b91d36c9aa86475da55437cae355477e668d" exitCode=0 Oct 30 18:31:11.704174 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:11.703761 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"499d97ae9783f008d05ee7812083b91d36c9aa86475da55437cae355477e668d"} Oct 30 18:31:12.448889 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:12.448857 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:12.449071 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:12.448860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:12.449071 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:12.448984 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:12.449071 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:12.448996 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:12.449225 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:12.449090 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:12.449225 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:12.449176 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:12.707052 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:12.706984 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"8ab46181f35125f040c3f8fe06f5bf06c53833100e7973ba3f4053ada175cb24"} Oct 30 18:31:13.712748 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.712557 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="379e0d8c424c85fedaa4436071f2d842fadbabeeed3b789c7cd12aa103ad8845" exitCode=0 Oct 30 18:31:13.713327 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.712644 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"379e0d8c424c85fedaa4436071f2d842fadbabeeed3b789c7cd12aa103ad8845"} Oct 30 18:31:13.718321 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718303 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:31:13.718668 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718638 2580 generic.go:356] "Generic (PLEG): container finished" podID="dbee4f62-3043-46ce-821a-b77f3746f83a" containerID="ac3e829195363f8fc97a94e0071d978d6f10bd577854aa276b3d57cc8a141d99" exitCode=1 Oct 30 18:31:13.718758 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718690 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"7b4a60d45a53b2b18a05eba18f385b6b287dcd371748239e6aa9b7cc073d36c3"} Oct 30 18:31:13.718807 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718797 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"af9e88d4be7441775af6470f82ff384af891f8065df7a73d48249b632261d4f4"} Oct 30 18:31:13.718864 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718850 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"e80a2e923107a2fb06e422b51cbd1e135bc49eb3b2d63994800ab58767feb9f0"} Oct 30 18:31:13.718916 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718904 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"4e8d20bbb265dd53ca898aee2f415ce55e42a74f461d0ee19e0a20845c13bfb1"} Oct 30 18:31:13.718965 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:13.718953 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerDied","Data":"ac3e829195363f8fc97a94e0071d978d6f10bd577854aa276b3d57cc8a141d99"} Oct 30 18:31:14.120585 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.120480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:14.120585 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.120565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:14.120813 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.120677 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:14.120813 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.120710 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:14.120813 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.120752 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:30.120733222 +0000 UTC m=+34.264798958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:14.120813 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.120774 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:30.120761679 +0000 UTC m=+34.264827408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:14.221846 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.221803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:14.222013 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.221963 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:14.222013 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.221984 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:14.222013 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.221995 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:14.222121 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.222043 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:31:30.222031753 +0000 UTC m=+34.366097495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:14.448468 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.448375 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:14.448468 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.448423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:14.448711 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.448515 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:14.448711 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.448564 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:14.448711 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.448617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:14.448848 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:14.448711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:14.723435 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.723401 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="c4cdc7708e720ffc60ed5079e16ef74181d79ef32e241735f56272e6c245049e" exitCode=0 Oct 30 18:31:14.724024 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:14.723468 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerDied","Data":"c4cdc7708e720ffc60ed5079e16ef74181d79ef32e241735f56272e6c245049e"} Oct 30 18:31:15.728407 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:15.728379 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:31:15.728844 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:15.728733 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"f83f3a395cb36d4fce592aa9ebe1a63fc885740b647c90267cbcb1996731ca3d"} Oct 30 18:31:15.731774 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:15.731754 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" event={"ID":"b3b05773-655d-4d2e-a924-2a6746e2a1e6","Type":"ContainerStarted","Data":"9d7fd86e11121604c2b3c159bf7be63faa69f6878842cb14ebd0da5508dd68d1"} Oct 30 18:31:15.811083 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:15.811026 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bpfcd" podStartSLOduration=9.547838289 podStartE2EDuration="19.81101267s" totalImagesPullingTime="10.263174381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:15.796535721 +0000 UTC m=+19.940601472" watchObservedRunningTime="2026-10-30 18:31:15.81101267 +0000 UTC m=+19.955078421" Oct 30 18:31:16.449350 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:16.449314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:16.449574 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:16.449399 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:16.449574 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:16.449505 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:16.449682 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:16.449635 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:16.449682 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:16.449670 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:16.449753 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:16.449729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:17.739138 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.738953 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:31:17.739694 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.739432 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"2af5e6fff6cff132bba17b5774283136785fa00c5db8fd27e91aae09bb3e15d1"} Oct 30 18:31:17.739694 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.739679 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:17.739781 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.739751 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:17.739840 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.739823 2580 scope.go:122] "RemoveContainer" containerID="ac3e829195363f8fc97a94e0071d978d6f10bd577854aa276b3d57cc8a141d99" Oct 30 18:31:17.756175 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:17.756153 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:18.453121 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.453085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:18.453477 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:18.453447 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:18.454032 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.454009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:18.454274 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:18.454252 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:18.455488 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.455466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:18.455711 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:18.455691 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:18.744973 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.744903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:31:18.745411 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.745250 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" event={"ID":"dbee4f62-3043-46ce-821a-b77f3746f83a","Type":"ContainerStarted","Data":"a3081a4b43e0e1caba7162f591db5f822acf23e5b878430cb0c9545f0a589e01"} Oct 30 18:31:18.745569 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.745525 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:18.760886 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.760865 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:18.846664 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:18.846615 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" podStartSLOduration=8.986173492 podStartE2EDuration="22.846601387s" totalImagesPullingTime="13.860427895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:18.84632539 +0000 UTC m=+22.990391143" watchObservedRunningTime="2026-10-30 18:31:18.846601387 +0000 UTC m=+22.990667148" Oct 30 18:31:19.376746 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.376665 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6kkd"] Oct 30 18:31:19.376890 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.376767 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:19.376890 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:19.376856 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:19.378347 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.378325 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-km5cq"] Oct 30 18:31:19.378458 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.378414 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:19.378514 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:19.378492 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:19.378918 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.378896 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x7rq7"] Oct 30 18:31:19.378998 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:19.378985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:19.379092 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:19.379074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:21.448470 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:21.448433 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:21.449060 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:21.448490 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:21.449189 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:21.448559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:21.449189 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:21.449135 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:21.449309 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:21.449252 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:21.449383 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:21.449364 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:22.800757 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:22.800728 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9r555_e025fb6d-2541-4243-a8d3-8fc4fc5f5940/dns-node-resolver/0.log" Oct 30 18:31:23.001771 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.001478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:23.204345 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.204315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:23.412974 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.412943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:23.448232 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.448194 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:23.448402 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.448244 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:23.448402 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.448245 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:23.448402 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:23.448310 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:23.448520 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:23.448404 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:23.448520 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:23.448464 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:23.653619 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.653371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:23.898836 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.898806 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fx675_cf7a8c1d-004b-423c-8d4c-27e2b6757aae/node-ca/0.log" Oct 30 18:31:23.974851 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:23.974809 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.448270 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:25.448241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:25.448764 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:25.448294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:25.448764 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:25.448395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:25.448764 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:25.448445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:25.448764 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:25.448556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:25.448764 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:25.448635 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:27.448877 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:27.448834 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:27.449266 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:27.448833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:27.449266 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:27.448960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:27.449266 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:27.449014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:27.449266 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:27.448852 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:27.449266 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:27.449108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:29.448472 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:29.448386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:29.448472 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:29.448418 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:29.448472 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:29.448428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:29.449022 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:29.448498 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:29.449022 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:29.448595 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:29.449022 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:29.448702 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:30.129800 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:30.129770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:30.129997 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:30.129827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:30.129997 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.129935 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:30.129997 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.129989 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs podName:b2780b90-9d95-44d8-bc10-8294255f08a5 nodeName:}" failed. No retries permitted until 2026-10-30 18:32:02.129976172 +0000 UTC m=+66.274041900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs") pod "network-metrics-daemon-km5cq" (UID: "b2780b90-9d95-44d8-bc10-8294255f08a5") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:30.130137 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.129935 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:30.130137 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.130069 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret podName:a56c3ed8-1972-402c-8195-b4dc842c69fb nodeName:}" failed. No retries permitted until 2026-10-30 18:32:02.130056983 +0000 UTC m=+66.274122726 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret") pod "global-pull-secret-syncer-t6kkd" (UID: "a56c3ed8-1972-402c-8195-b4dc842c69fb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:30.230759 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:30.230727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:30.230924 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.230842 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:30.230924 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.230856 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:30.230924 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.230865 2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7gmg for pod openshift-network-diagnostics/network-check-target-x7rq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:30.230924 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:30.230909 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg podName:195ffa97-91dc-4cad-802d-97027141794c nodeName:}" failed. No retries permitted until 2026-10-30 18:32:02.230897229 +0000 UTC m=+66.374962972 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7gmg" (UniqueName: "kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg") pod "network-check-target-x7rq7" (UID: "195ffa97-91dc-4cad-802d-97027141794c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:31.448341 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:31.448310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:31.448748 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:31.448310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:31.448748 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:31.448414 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:31.448748 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:31.448310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:31.448748 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:31.448481 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:31.448748 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:31.448575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:33.448139 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:33.448097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:33.448139 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:33.448132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:33.448702 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:33.448199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:33.448702 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:33.448213 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:33.448702 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:33.448652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:33.448847 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:33.448519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:35.449073 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:35.449035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:35.449073 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:35.449062 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:35.449629 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:35.449074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:35.449629 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:35.449143 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:35.449629 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:35.449232 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:35.449629 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:35.449321 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:37.448992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:37.448958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:37.449365 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:37.449016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:37.449365 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:37.449102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:37.449365 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:37.449119 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:37.449365 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:37.449176 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:37.449365 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:37.449223 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:39.448992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:39.448953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:39.448992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:39.448974 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:39.448992 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:39.448977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:39.449563 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:39.449087 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:39.449563 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:39.449189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:39.449563 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:39.449257 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:41.449005 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.448960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:41.449005 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.449007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:41.449495 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.449066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:41.449495 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:41.449075 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-km5cq" podUID="b2780b90-9d95-44d8-bc10-8294255f08a5" Oct 30 18:31:41.449495 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:41.449137 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x7rq7" podUID="195ffa97-91dc-4cad-802d-97027141794c" Oct 30 18:31:41.449495 ip-10-0-136-105 kubenswrapper[2580]: E1030 18:31:41.449225 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6kkd" podUID="a56c3ed8-1972-402c-8195-b4dc842c69fb" Oct 30 18:31:41.695850 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.695781 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeReady" Oct 30 18:31:41.696019 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.695920 2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:31:41.774166 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.774128 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7fvzb"] Oct 30 18:31:41.777041 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.777019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:41.782754 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.782724 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w9swt"] Oct 30 18:31:41.787748 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.787726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.826413 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.826382 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fvzb"] Oct 30 18:31:41.826957 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.826936 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w9swt"] Oct 30 18:31:41.834075 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.834051 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c8dqv"] Oct 30 18:31:41.837088 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.837069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:41.853462 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853440 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853766 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853808 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853818 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853819 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853887 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:31:41.854132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.853881 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:31:41.854422 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.854174 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:31:41.854422 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.854384 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:31:41.874395 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.874373 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c8dqv"] Oct 30 18:31:41.910779 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2pws\" (UniqueName: \"kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.910948 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.910948 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfz84\" (UniqueName: \"kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:41.910948 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.910948 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910898 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.910948 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:41.911136 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.910951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:41.952824 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.952795 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:31:41.952824 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.952812 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:31:41.953028 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:41.952849 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:31:42.012224 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.012411 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.012411 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:42.012517 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2pws\" (UniqueName: \"kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.012517 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.012642 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g95f\" (UniqueName: \"kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.012642 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.012726 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz84\" (UniqueName: \"kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:42.012726 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.012726 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.012850 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.012945 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012927 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.013006 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.012955 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.013310 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.013290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.016217 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.016189 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.016303 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.016237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:42.039940 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.039912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfz84\" (UniqueName: \"kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:42.043406 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.043382 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2pws\" (UniqueName: \"kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.090370 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.090334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:31:42.102603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.102428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:31:42.113594 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.113566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.113692 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.113609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.113692 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.113635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.113692 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.113662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g95f\" (UniqueName: \"kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.113977 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.113956 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.114246 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.114217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.115923 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.115902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.138916 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.138880 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g95f\" (UniqueName: \"kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.149870 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.149840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:42.420642 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.420610 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fvzb"] Oct 30 18:31:42.423991 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.423914 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w9swt"] Oct 30 18:31:42.426589 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:42.426565 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7fe3f2_f764_4843_8732_f5434a5f82cc.slice/crio-2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29 WatchSource:0}: Error finding container 2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29: Status 404 returned error can't find the container with id 2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29 Oct 30 18:31:42.462305 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.462279 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c8dqv"] Oct 30 18:31:42.465075 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:42.465047 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502b8dd3_61d1_4a04_883e_35cf6460a661.slice/crio-560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb WatchSource:0}: Error finding container 560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb: Status 404 returned error can't find the container with id 560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb Oct 30 18:31:42.797757 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.797687 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fvzb" event={"ID":"11357dc0-5895-4a3d-9969-209df38fa006","Type":"ContainerStarted","Data":"ef2e7926dac7fd3af94a9537629f328ea6ee5e70364a782721ac01cbd193a3ef"} Oct 30 18:31:42.798831 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.798809 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c8dqv" event={"ID":"502b8dd3-61d1-4a04-883e-35cf6460a661","Type":"ContainerStarted","Data":"560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb"} Oct 30 18:31:42.800216 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.800199 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9swt" event={"ID":"0c7fe3f2-f764-4843-8732-f5434a5f82cc","Type":"ContainerStarted","Data":"5c62238aa8ed625ef58ebea975c1d1e1400709869ee4e33d0761d6219aefbc6c"} Oct 30 18:31:42.800275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:42.800238 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9swt" event={"ID":"0c7fe3f2-f764-4843-8732-f5434a5f82cc","Type":"ContainerStarted","Data":"2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29"} Oct 30 18:31:43.448295 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.448259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:31:43.448445 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.448259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:31:43.448523 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.448278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:31:43.480386 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480358 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:31:43.480386 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480386 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:31:43.480891 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480402 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:31:43.480891 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480446 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:31:43.480891 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480457 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:31:43.480891 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480464 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:31:43.480891 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.480510 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:31:43.807300 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:43.807213 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9swt" event={"ID":"0c7fe3f2-f764-4843-8732-f5434a5f82cc","Type":"ContainerStarted","Data":"5e551b8562983141f0241026a872ee12ed2c0cc166980551cebb251b7d24d828"} Oct 30 18:31:44.812656 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.812620 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c8dqv" event={"ID":"502b8dd3-61d1-4a04-883e-35cf6460a661","Type":"ContainerStarted","Data":"afc2615a4ed99628212c100fa7a228c4ab1451f1e9f51946d572834990319bd8"} Oct 30 18:31:44.813174 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.812675 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c8dqv" event={"ID":"502b8dd3-61d1-4a04-883e-35cf6460a661","Type":"ContainerStarted","Data":"28a49178f0ebfdb7ed225a2c8465f4dd55d9645be9c065843dd3eafc366b59cf"} Oct 30 18:31:44.813174 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.812755 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:44.815250 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.815221 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w9swt" event={"ID":"0c7fe3f2-f764-4843-8732-f5434a5f82cc","Type":"ContainerStarted","Data":"377fd3a06ef4798173fbdaab9fd22b88b9dd9eb71256349e9cd0850a4b2bf20b"} Oct 30 18:31:44.860695 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.860639 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-c8dqv" podStartSLOduration=2.091512596 podStartE2EDuration="3.860622369s" totalImagesPullingTime="1.769109773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:44.859576864 +0000 UTC m=+49.003642634" watchObservedRunningTime="2026-10-30 18:31:44.860622369 +0000 UTC m=+49.004688120" Oct 30 18:31:44.906939 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.906880 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w9swt" podStartSLOduration=1.775465723 podStartE2EDuration="3.90686371s" totalImagesPullingTime="2.131397987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:44.90640918 +0000 UTC m=+49.050474931" watchObservedRunningTime="2026-10-30 18:31:44.90686371 +0000 UTC m=+49.050929460" Oct 30 18:31:44.966887 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:44.966817 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c8dqv_502b8dd3-61d1-4a04-883e-35cf6460a661/kube-rbac-proxy/0.log" Oct 30 18:31:45.819465 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:45.819427 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fvzb" event={"ID":"11357dc0-5895-4a3d-9969-209df38fa006","Type":"ContainerStarted","Data":"01b8f8a3d4c9e657fbf049630abf856e969026f8a2aaa8315d965d950397dc79"} Oct 30 18:31:45.864042 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:45.863991 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7fvzb" podStartSLOduration=2.243919749 podStartE2EDuration="4.863978934s" totalImagesPullingTime="2.620059185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:45.863387785 +0000 UTC m=+50.007453536" watchObservedRunningTime="2026-10-30 18:31:45.863978934 +0000 UTC m=+50.008044684" Oct 30 18:31:45.969746 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:45.969712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9r555_e025fb6d-2541-4243-a8d3-8fc4fc5f5940/dns-node-resolver/0.log" Oct 30 18:31:47.366037 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.366010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fx675_cf7a8c1d-004b-423c-8d4c-27e2b6757aae/node-ca/0.log" Oct 30 18:31:47.565655 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.565623 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:31:47.568401 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.568386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.587134 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.587107 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 18:31:47.601467 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601441 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 18:31:47.601467 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601441 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 18:31:47.601687 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601559 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 18:31:47.601687 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601578 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 18:31:47.601687 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601611 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s77r6\"" Oct 30 18:31:47.601687 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601619 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 18:31:47.601687 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.601625 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 18:31:47.630712 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.630641 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 18:31:47.631519 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.631501 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:31:47.760602 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760745 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760745 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760745 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760840 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760840 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760896 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.760927 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.760900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861558 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861727 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861727 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861727 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861727 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861915 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861915 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.861915 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.861875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.862278 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.862252 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.862482 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.862465 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.862568 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.862481 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.862641 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.862619 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.862758 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.862743 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.864660 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.864639 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.864745 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.864672 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.887125 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.887083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:47.979011 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:47.978984 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7fvzb_11357dc0-5895-4a3d-9969-209df38fa006/serve-healthcheck-canary/0.log" Oct 30 18:31:48.178872 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.178791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:31:48.503923 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.503893 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:31:48.507831 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:48.507800 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6487588c_2589_404f_8711_3ccf6a999ebb.slice/crio-3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96 WatchSource:0}: Error finding container 3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96: Status 404 returned error can't find the container with id 3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96 Oct 30 18:31:48.826707 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.826625 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff767d568-st4n4" event={"ID":"6487588c-2589-404f-8711-3ccf6a999ebb","Type":"ContainerStarted","Data":"3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96"} Oct 30 18:31:48.902715 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.902681 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb"] Oct 30 18:31:48.906875 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.906859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:48.933018 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.932995 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:31:48.947139 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.947102 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 18:31:48.947308 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.947154 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:31:48.947391 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.947318 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:31:48.947391 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.947320 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:31:48.947484 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.947412 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7lgxg\"" Oct 30 18:31:48.954482 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:48.954450 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb"] Oct 30 18:31:49.004165 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.004140 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-24l9n"] Oct 30 18:31:49.007070 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.007056 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.048268 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.048245 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:31:49.048490 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.048454 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:31:49.048664 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.048647 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twcvb\"" Oct 30 18:31:49.048859 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.048846 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:31:49.070501 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.070473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.070655 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.070506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh4g6\" (UniqueName: \"kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.070655 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.070567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.070780 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.070666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.171934 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.171857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.171934 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.171907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172130 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.171973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhhhq\" (UniqueName: \"kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172130 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh4g6\" (UniqueName: \"kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.172130 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172239 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.172275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.172310 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172350 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172391 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172529 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172508 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.172627 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.172627 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.172616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.173074 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.173051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.174727 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.174698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.174828 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.174722 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.197024 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.197000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh4g6\" (UniqueName: \"kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.221897 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.221874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:31:49.273858 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.273792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.273858 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.273843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274044 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.273986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274080 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274111 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274197 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274197 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274384 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274365 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.274634 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.274608 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.275304 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.275038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.275304 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.275040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhhhq\" (UniqueName: \"kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.275304 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.275147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.275304 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.275270 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.275829 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.275803 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.280435 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.280412 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.281132 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.281106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.304489 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.304457 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhhhq\" (UniqueName: \"kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.321482 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.321451 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:31:49.333986 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:49.333948 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc7152c_07ad_466c_aed1_1e289c384fde.slice/crio-2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae WatchSource:0}: Error finding container 2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae: Status 404 returned error can't find the container with id 2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae Oct 30 18:31:49.551900 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.551869 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb"] Oct 30 18:31:49.555343 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:49.555314 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd5c23b_225b_4eff_8b03_77fe7c04e56c.slice/crio-8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527 WatchSource:0}: Error finding container 8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527: Status 404 returned error can't find the container with id 8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527 Oct 30 18:31:49.832640 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.832559 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24l9n" event={"ID":"3dc7152c-07ad-466c-aed1-1e289c384fde","Type":"ContainerStarted","Data":"2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae"} Oct 30 18:31:49.834603 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.834576 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" event={"ID":"4bd5c23b-225b-4eff-8b03-77fe7c04e56c","Type":"ContainerStarted","Data":"d11947b6d6a0c7c2882e11a8e85d0df8f1541ce7c4e74f8205c5522536b3f564"} Oct 30 18:31:49.834775 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.834635 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" event={"ID":"4bd5c23b-225b-4eff-8b03-77fe7c04e56c","Type":"ContainerStarted","Data":"c776488756ef4e0da68b501c1db7298f197d098337dc409fbea3d1961965ebc1"} Oct 30 18:31:49.834775 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:49.834662 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" event={"ID":"4bd5c23b-225b-4eff-8b03-77fe7c04e56c","Type":"ContainerStarted","Data":"8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527"} Oct 30 18:31:50.760818 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:50.760787 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:31:50.841030 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:50.840999 2580 generic.go:356] "Generic (PLEG): container finished" podID="3dc7152c-07ad-466c-aed1-1e289c384fde" containerID="e09b837868e685eede2e3696410f7f24c651bf1fde47486cf1fbf49b749a6f16" exitCode=0 Oct 30 18:31:50.841192 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:50.841051 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24l9n" event={"ID":"3dc7152c-07ad-466c-aed1-1e289c384fde","Type":"ContainerDied","Data":"e09b837868e685eede2e3696410f7f24c651bf1fde47486cf1fbf49b749a6f16"} Oct 30 18:31:51.065959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.065882 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7"] Oct 30 18:31:51.069607 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.069582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089705 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089727 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089783 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089730 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089832 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089899 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089727 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089917 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5se1021gu1ucd\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089983 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r2wdj\"" Oct 30 18:31:51.090021 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.089999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt8jb\" (UniqueName: \"kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.096073 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.096048 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7"] Oct 30 18:31:51.190778 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.190959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.190959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.190959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.190959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.191143 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt8jb\" (UniqueName: \"kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.191143 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.190998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.191143 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.191027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.191143 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.191072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.192099 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.191812 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.192626 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.192576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.194699 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.194663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.195014 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.194991 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.195091 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.195027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.195172 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.195151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.195423 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.195406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.196955 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.196936 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.223942 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.223915 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt8jb\" (UniqueName: \"kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.396559 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.396462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:51.744853 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.744819 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7"] Oct 30 18:31:51.747733 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:51.747708 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2051f0_a1e2_4a95_a7a6_9a6d45c01c0b.slice/crio-d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262 WatchSource:0}: Error finding container d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262: Status 404 returned error can't find the container with id d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262 Oct 30 18:31:51.848957 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.848920 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff767d568-st4n4" event={"ID":"6487588c-2589-404f-8711-3ccf6a999ebb","Type":"ContainerStarted","Data":"45bcca9b5d9258f9fe4597278f6bd66f0cbbd9f82ea59b470b266186fd148a9f"} Oct 30 18:31:51.851141 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.851107 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24l9n" event={"ID":"3dc7152c-07ad-466c-aed1-1e289c384fde","Type":"ContainerStarted","Data":"7877fbddb135297f63abe21131b1ed0b420f0fbd45e848c82fbb310c22e1c4a5"} Oct 30 18:31:51.851256 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.851164 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24l9n" event={"ID":"3dc7152c-07ad-466c-aed1-1e289c384fde","Type":"ContainerStarted","Data":"56602831d07e8fb8ecf36b9bbc4922b674cd1d1206026eeefaf6f96ecb719ca3"} Oct 30 18:31:51.853071 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.853042 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" event={"ID":"4bd5c23b-225b-4eff-8b03-77fe7c04e56c","Type":"ContainerStarted","Data":"2a7e3f51650a0b62c82465f4e171af76af47d23500c6d3ff2b687397009064f3"} Oct 30 18:31:51.854060 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.854039 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262"} Oct 30 18:31:51.902802 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.902753 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6ff767d568-st4n4" podStartSLOduration=1.8204453699999998 podStartE2EDuration="4.902740077s" totalImagesPullingTime="3.082294707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:51.901413812 +0000 UTC m=+56.045479563" watchObservedRunningTime="2026-10-30 18:31:51.902740077 +0000 UTC m=+56.046805828" Oct 30 18:31:51.947583 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:51.947515 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" podStartSLOduration=2.8332787699999997 podStartE2EDuration="3.947500994s" totalImagesPullingTime="1.114222224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:51.947388428 +0000 UTC m=+56.091454184" watchObservedRunningTime="2026-10-30 18:31:51.947500994 +0000 UTC m=+56.091566738" Oct 30 18:31:52.029734 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:52.029644 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-24l9n" podStartSLOduration=2.874406329 podStartE2EDuration="4.029630012s" totalImagesPullingTime="1.07954347s" totalInitContainerRuntime="75.680213ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:52.02847329 +0000 UTC m=+56.172539040" watchObservedRunningTime="2026-10-30 18:31:52.029630012 +0000 UTC m=+56.173695769" Oct 30 18:31:53.461306 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.461281 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7969db6d87-blhz9"] Oct 30 18:31:53.465650 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.465633 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.483700 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.483669 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 18:31:53.483700 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.483686 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 18:31:53.487565 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.487530 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 18:31:53.496370 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.496331 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vz2m7\"" Oct 30 18:31:53.496474 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.496397 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 18:31:53.496740 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.496724 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6harcg9ua62ce\"" Oct 30 18:31:53.498934 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.498916 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7969db6d87-blhz9"] Oct 30 18:31:53.511638 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511732 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511732 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511816 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4mx\" (UniqueName: \"kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511845 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511872 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.511944 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.511913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.612845 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.612819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.612970 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.612865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.612970 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.612925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.612970 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.612963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.613119 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.612998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.613119 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.613048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4mx\" (UniqueName: \"kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.613119 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.613085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.614239 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.613746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.614239 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.613872 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.614479 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.614461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.616214 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.616188 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.616404 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.616384 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.616557 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.616482 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.639770 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.639743 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4mx\" (UniqueName: \"kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.846406 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.846319 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:31:53.862689 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.862652 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"612fbf7fcd02ffab7bb288b0128a8a9340a574cc3f1b4e360b03a2d5c86afbc9"} Oct 30 18:31:53.862790 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.862775 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"3529da6d3c78a92a6192be259bfcbdc4f3cfe7a9fdab1e19cec0b362cd550e19"} Oct 30 18:31:53.862836 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:53.862824 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"9f866017d017e798e32069c7fe69899bf7f7b96ed0b9b8e2aa94885cbb62515d"} Oct 30 18:31:54.174643 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.174561 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7969db6d87-blhz9"] Oct 30 18:31:54.180195 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:54.180168 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852071ed_f1b5_4b6e_8ef5_1deb88d86855.slice/crio-f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d WatchSource:0}: Error finding container f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d: Status 404 returned error can't find the container with id f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d Oct 30 18:31:54.822293 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.822191 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c8dqv" Oct 30 18:31:54.870434 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.870397 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"30397ec7b1096ea3bd1d1c37e639123121405e1f5abfcc57d9849538fa3a2646"} Oct 30 18:31:54.870629 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.870465 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"d484f2f31a7e56c702079bd6962d8d29edc7ecd4d265e0a79c0926559b878382"} Oct 30 18:31:54.870629 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.870506 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" event={"ID":"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b","Type":"ContainerStarted","Data":"fbafb2907902ebdacdc6424ed844df7f62f649afa94779e76d35fc7aea1e6d24"} Oct 30 18:31:54.870629 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.870564 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:31:54.872214 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.872188 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" event={"ID":"852071ed-f1b5-4b6e-8ef5-1deb88d86855","Type":"ContainerStarted","Data":"f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d"} Oct 30 18:31:54.923482 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:54.922925 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" podStartSLOduration=1.409861289 podStartE2EDuration="3.922904441s" totalImagesPullingTime="2.513043152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:54.922905071 +0000 UTC m=+59.066970825" watchObservedRunningTime="2026-10-30 18:31:54.922904441 +0000 UTC m=+59.066970195" Oct 30 18:31:55.877137 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.877043 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" event={"ID":"852071ed-f1b5-4b6e-8ef5-1deb88d86855","Type":"ContainerStarted","Data":"401573016fc289c97a15ccb30ec4e829e8d7fda2da2a5d42a5f15d33442fe3f8"} Oct 30 18:31:55.902429 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.902400 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:31:55.908654 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.908617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.926958 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.926931 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:31:55.927090 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.926991 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:31:55.927130 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.927111 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:31:55.931357 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931453 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931453 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931453 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931681 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931722 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931767 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931821 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931923 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.931985 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.931965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932066 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932191 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932225 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932225 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932303 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932361 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932408 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932448 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932729 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.932876 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.932853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:55.939816 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.939797 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:31:55.939985 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.939946 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:31:55.968356 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.968333 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:31:55.968508 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.968493 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:31:55.970039 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.970021 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:31:55.970496 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.970464 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" podStartSLOduration=1.60483647 podStartE2EDuration="2.970454287s" totalImagesPullingTime="1.365617817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:55.939499225 +0000 UTC m=+60.083564975" watchObservedRunningTime="2026-10-30 18:31:55.970454287 +0000 UTC m=+60.114520050" Oct 30 18:31:55.971338 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:55.971324 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:31:56.031879 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031817 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:31:56.032049 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031913 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:31:56.032049 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031914 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bqjtk\"" Oct 30 18:31:56.032049 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031914 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:31:56.032049 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031920 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:31:56.032049 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.031971 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:31:56.032215 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.032054 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:31:56.032215 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.032129 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8lqh3egd5j83h\"" Oct 30 18:31:56.033262 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033305 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033351 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033351 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033351 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033555 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033695 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033800 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033800 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033800 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033752 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033800 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.033993 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.033984 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034043 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.034275 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.034133 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.035932 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.035907 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.036557 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.036520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.036982 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.036957 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.037159 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.037141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.037221 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.037202 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.037371 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.037350 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.037449 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.037375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.037872 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.037851 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.038028 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.038003 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.038093 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.038027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.038278 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.038248 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.038884 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.038858 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.038974 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.038911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.039434 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.039398 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.039621 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.039605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.040108 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.040090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.062104 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.062083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.233310 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.233277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:31:56.407364 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.407331 2580 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:31:56.560929 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.560903 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:31:56.562853 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:31:56.562827 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19921861_8bf5_4a85_b934_56a1b07cf4ae.slice/crio-a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2 WatchSource:0}: Error finding container a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2: Status 404 returned error can't find the container with id a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2 Oct 30 18:31:56.881796 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:56.881718 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2"} Oct 30 18:31:57.886285 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:57.886183 2580 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" exitCode=0 Oct 30 18:31:57.886285 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:57.886254 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} Oct 30 18:31:58.179385 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:31:58.179311 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:00.884197 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:00.884170 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:02.159175 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.159143 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:32:02.167025 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.166904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.183207 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.183032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:02.183207 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.183102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:02.223917 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.223895 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:32:02.224019 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.223970 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:32:02.226690 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.226670 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:02.227014 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.226934 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:02.234988 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.233526 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:32:02.287967 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.287931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288148 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288261 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288360 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:02.288465 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288576 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288684 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288684 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.288805 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.288766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.307379 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.307351 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:32:02.324981 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.324959 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:32:02.333213 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.333184 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:02.379360 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.379303 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:32:02.383798 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.383781 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:02.389828 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.389897 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.389897 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.389959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.389959 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.390016 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.389974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.390197 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.390044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.390197 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.390089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.390609 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.390588 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:32:02.390760 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.390737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.391063 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.391042 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.391187 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.391148 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.391801 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.391713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.392119 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.392061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.393627 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.393610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.393708 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.393625 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.394885 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.394796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:02.394949 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.394931 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:32:02.400619 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.400601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:02.417388 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.417361 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.481000 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.480706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:02.696694 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.696666 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6kkd"] Oct 30 18:32:02.699469 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:32:02.699439 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56c3ed8_1972_402c_8195_b4dc842c69fb.slice/crio-bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0 WatchSource:0}: Error finding container bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0: Status 404 returned error can't find the container with id bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0 Oct 30 18:32:02.730568 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.727064 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x7rq7"] Oct 30 18:32:02.740532 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.740510 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-km5cq"] Oct 30 18:32:02.742788 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:32:02.742761 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2780b90_9d95_44d8_bc10_8294255f08a5.slice/crio-16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd WatchSource:0}: Error finding container 16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd: Status 404 returned error can't find the container with id 16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd Oct 30 18:32:02.789003 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.788973 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:32:02.794824 ip-10-0-136-105 kubenswrapper[2580]: W1030 18:32:02.794801 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9150d1_1c80_4155_9b40_64793e254389.slice/crio-000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2 WatchSource:0}: Error finding container 000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2: Status 404 returned error can't find the container with id 000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2 Oct 30 18:32:02.905257 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.905221 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x7rq7" event={"ID":"195ffa97-91dc-4cad-802d-97027141794c","Type":"ContainerStarted","Data":"8a38dc92dc611e4aeb4cb4e5d9f215ae93ae14f15bfdbec4b676b6b5329146b2"} Oct 30 18:32:02.906609 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.906530 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6kkd" event={"ID":"a56c3ed8-1972-402c-8195-b4dc842c69fb","Type":"ContainerStarted","Data":"bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0"} Oct 30 18:32:02.910163 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910107 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} Oct 30 18:32:02.910322 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910175 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} Oct 30 18:32:02.910322 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910247 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} Oct 30 18:32:02.910322 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910289 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} Oct 30 18:32:02.910473 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910326 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} Oct 30 18:32:02.910473 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.910372 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerStarted","Data":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} Oct 30 18:32:02.911963 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.911940 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcd65b88-x42qd" event={"ID":"8a9150d1-1c80-4155-9b40-64793e254389","Type":"ContainerStarted","Data":"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771"} Oct 30 18:32:02.912055 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.911976 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcd65b88-x42qd" event={"ID":"8a9150d1-1c80-4155-9b40-64793e254389","Type":"ContainerStarted","Data":"000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2"} Oct 30 18:32:02.913231 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.913209 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-km5cq" event={"ID":"b2780b90-9d95-44d8-bc10-8294255f08a5","Type":"ContainerStarted","Data":"16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd"} Oct 30 18:32:02.985368 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:02.985314 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.92720439 podStartE2EDuration="7.985299645s" totalImagesPullingTime="5.044598992s" totalInitContainerRuntime="13.496263ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:02.981769988 +0000 UTC m=+67.125835738" watchObservedRunningTime="2026-10-30 18:32:02.985299645 +0000 UTC m=+67.129365399" Oct 30 18:32:03.044867 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:03.044784 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7cbcd65b88-x42qd" podStartSLOduration=1.044768788 podStartE2EDuration="1.044768788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:03.044337433 +0000 UTC m=+67.188403183" watchObservedRunningTime="2026-10-30 18:32:03.044768788 +0000 UTC m=+67.188834538" Oct 30 18:32:04.925758 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:04.925724 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-km5cq" event={"ID":"b2780b90-9d95-44d8-bc10-8294255f08a5","Type":"ContainerStarted","Data":"366562f34d3ca51b3af1b84e19d04afb17d5633257643466cb356c90783c4736"} Oct 30 18:32:04.926259 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:04.925786 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-km5cq" event={"ID":"b2780b90-9d95-44d8-bc10-8294255f08a5","Type":"ContainerStarted","Data":"224d96ef96c5a1a222b7f7dbc6a5cdc02fd861372babad9602e93f53f78e4db2"} Oct 30 18:32:05.016686 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:05.016623 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-km5cq" podStartSLOduration=67.774338432 podStartE2EDuration="1m9.016602149s" totalImagesPullingTime="1.242263717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:05.015401734 +0000 UTC m=+69.159467485" watchObservedRunningTime="2026-10-30 18:32:05.016602149 +0000 UTC m=+69.160667899" Oct 30 18:32:06.234060 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:06.234022 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:06.935226 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:06.935138 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x7rq7" event={"ID":"195ffa97-91dc-4cad-802d-97027141794c","Type":"ContainerStarted","Data":"4a872739c2bc9a0390e872a1735153be43d6ffbb562c4a6a9779d77a85847429"} Oct 30 18:32:06.935392 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:06.935267 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:06.981117 ip-10-0-136-105 kubenswrapper[2580]: I1030 18:32:06.981061 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x7rq7" podStartSLOduration=67.039766957 podStartE2EDuration="1m10.981042766s" totalImagesPullingTime="3.941275809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:06.980262505 +0000 UTC m=+71.124328256" watchObservedRunningTime="2026-10-30 18:32:06.981042766 +0000 UTC m=+71.125108518" Oct 30 18:32:08.497894 ip-10-0-136-105 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:32:08.508957 ip-10-0-136-105 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:32:08.509122 ip-10-0-136-105 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:32:08.509169 ip-10-0-136-105 systemd[1]: kubelet.service: Consumed 4.184s CPU time. Oct 30 18:32:08.523238 ip-10-0-136-105 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:32:08.584120 ip-10-0-136-105 kubenswrapper[6492]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:08.584120 ip-10-0-136-105 kubenswrapper[6492]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:32:08.584120 ip-10-0-136-105 kubenswrapper[6492]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:08.584120 ip-10-0-136-105 kubenswrapper[6492]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:08.586433 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586340 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.586433 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586431 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.586433 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586436 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586440 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586443 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586446 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586449 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586452 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586455 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586458 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586461 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586464 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586467 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586469 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586472 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586475 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586478 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586481 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586484 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586486 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586489 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.586572 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586491 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586494 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586497 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586500 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586504 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586507 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586519 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586524 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586527 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586530 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586533 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586560 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586563 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586566 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586569 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586572 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586575 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586577 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586580 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586583 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.587141 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586586 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586588 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586591 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586594 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586597 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586602 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586605 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586608 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586611 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586614 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586617 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586620 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586622 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586625 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586628 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586632 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586636 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586640 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586642 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.587685 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586646 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586649 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586652 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586654 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586657 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586660 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586662 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586665 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586668 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586670 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586673 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586676 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586679 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586681 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586684 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586687 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586689 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586692 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586696 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586699 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.588132 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586702 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586704 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586707 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586709 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586712 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586716 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586718 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586721 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586726 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586729 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586731 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586734 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586736 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586740 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586742 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586745 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586748 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586750 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586753 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.588658 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586755 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586758 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586761 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586763 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586766 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586768 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.586771 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587569 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587576 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587579 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587582 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587587 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587590 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587592 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587595 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587598 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587600 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587603 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587606 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587608 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.589121 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587611 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587615 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587618 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587620 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587623 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587625 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587628 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587631 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587636 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587639 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587642 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587644 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587647 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587650 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587652 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587655 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587657 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587660 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587663 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.589613 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587665 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587667 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587670 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587672 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587677 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587679 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587682 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587684 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587687 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587690 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587693 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587695 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587698 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587701 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587704 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587707 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587709 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587712 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587715 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587717 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.590093 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587720 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587723 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587727 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587731 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587735 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587738 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587741 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587744 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587747 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587750 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587752 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587755 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587757 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587760 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587763 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587766 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587769 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587771 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587774 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.590624 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587776 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587779 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587781 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587784 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587786 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587789 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587791 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587794 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587796 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587799 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587802 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587804 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587807 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587809 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587811 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587815 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587817 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587820 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587823 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587825 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.591085 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587828 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587831 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587833 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587836 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587838 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587842 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587845 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587847 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587850 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587852 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587854 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587857 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587859 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587862 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.587864 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588346 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588351 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588354 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588357 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588360 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.591594 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588363 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588366 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588368 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588372 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588375 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588377 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588380 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588383 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588386 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588388 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588391 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588394 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588397 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588400 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588402 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588405 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588407 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588410 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588413 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588415 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.592065 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588418 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588420 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588423 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588425 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588427 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588430 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588432 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588435 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588437 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588440 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588444 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588448 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588451 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588454 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588456 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588459 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588461 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588464 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588466 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.592589 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588471 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588474 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588477 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588479 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588482 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588484 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588487 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588490 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588492 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588495 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588497 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588500 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588505 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588507 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588510 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588512 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588515 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588517 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588521 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588524 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.593044 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588527 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588529 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588532 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588535 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588552 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588554 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588557 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588559 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588562 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588565 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588567 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588570 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588575 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588577 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588581 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588584 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588586 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588589 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588591 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.593515 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588594 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588597 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588599 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588602 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588604 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588608 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588611 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588613 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588616 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588618 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588621 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588623 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588626 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588628 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588631 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588634 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588637 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588639 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588642 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588644 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.593971 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588648 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588651 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.588653 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588743 6492 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588751 6492 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588759 6492 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588764 6492 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588768 6492 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588772 6492 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588777 6492 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588781 6492 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588784 6492 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588787 6492 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588791 6492 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588794 6492 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588797 6492 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588800 6492 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588805 6492 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588808 6492 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588811 6492 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588814 6492 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588818 6492 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588821 6492 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:32:08.594473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588824 6492 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588827 6492 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588831 6492 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588835 6492 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588838 6492 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588840 6492 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588844 6492 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588847 6492 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588850 6492 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588852 6492 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588856 6492 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588859 6492 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588863 6492 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588867 6492 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588871 6492 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588874 6492 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588877 6492 flags.go:64] FLAG: --enable-server="true" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588880 6492 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588886 6492 flags.go:64] FLAG: --event-burst="100" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588889 6492 flags.go:64] FLAG: --event-qps="50" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588892 6492 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588896 6492 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588898 6492 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588903 6492 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:32:08.595062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588906 6492 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588909 6492 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588914 6492 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588917 6492 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588920 6492 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588923 6492 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588926 6492 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588929 6492 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588932 6492 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588935 6492 flags.go:64] FLAG: --feature-gates="" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588939 6492 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588942 6492 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588945 6492 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588948 6492 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588951 6492 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588955 6492 flags.go:64] FLAG: --help="false" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588957 6492 flags.go:64] FLAG: --hostname-override="ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588961 6492 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588964 6492 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588967 6492 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588970 6492 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588973 6492 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588978 6492 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:32:08.596091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588981 6492 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588983 6492 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588986 6492 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588989 6492 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588993 6492 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588996 6492 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.588999 6492 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589003 6492 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589006 6492 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589008 6492 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589011 6492 flags.go:64] FLAG: --lock-file="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589016 6492 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589019 6492 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589022 6492 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589027 6492 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589030 6492 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589034 6492 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589036 6492 flags.go:64] FLAG: --logging-format="text" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589040 6492 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589043 6492 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589046 6492 flags.go:64] FLAG: --manifest-url="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589049 6492 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589053 6492 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589056 6492 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589060 6492 flags.go:64] FLAG: --max-pods="110" Oct 30 18:32:08.596671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589063 6492 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589066 6492 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589069 6492 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589072 6492 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589075 6492 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589078 6492 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589082 6492 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589088 6492 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589091 6492 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589094 6492 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589097 6492 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589100 6492 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589106 6492 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589109 6492 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589113 6492 flags.go:64] FLAG: --port="10250" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589116 6492 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589119 6492 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0173b6aac6f7db1b0" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589122 6492 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589127 6492 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589130 6492 flags.go:64] FLAG: --register-node="true" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589133 6492 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589137 6492 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589140 6492 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589143 6492 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589146 6492 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589150 6492 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:32:08.597368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589153 6492 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589156 6492 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589159 6492 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589162 6492 flags.go:64] FLAG: --runonce="false" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589165 6492 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589168 6492 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589171 6492 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589174 6492 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589177 6492 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589180 6492 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589183 6492 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589186 6492 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589190 6492 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589193 6492 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589196 6492 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589199 6492 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589202 6492 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589205 6492 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589208 6492 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589213 6492 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589216 6492 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589219 6492 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589223 6492 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589226 6492 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589230 6492 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:32:08.598067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589233 6492 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589236 6492 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589239 6492 flags.go:64] FLAG: --v="2" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589243 6492 flags.go:64] FLAG: --version="false" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589247 6492 flags.go:64] FLAG: --vmodule="" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589251 6492 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589254 6492 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589382 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589386 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589389 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589394 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589399 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589402 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589404 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589407 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589410 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589413 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589416 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589418 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589422 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589424 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.598721 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589427 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589430 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589433 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589435 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589438 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589441 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589444 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589447 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589451 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589454 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589458 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589460 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589464 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589466 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589469 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589471 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589474 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589476 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589479 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.599222 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589481 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589484 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589486 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589489 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589491 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589494 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589496 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589499 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589501 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589504 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589507 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589511 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589513 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589516 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589519 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589521 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589524 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589526 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589529 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.599709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589531 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589534 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589551 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589554 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589558 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589561 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589564 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589566 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589569 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589571 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589574 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589576 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589579 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589582 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589584 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589587 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589589 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589591 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589594 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589597 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.600192 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589599 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589601 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589604 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589606 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589610 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589613 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589616 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589619 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589622 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589624 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589627 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589629 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589632 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589634 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589638 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589641 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589645 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589647 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589650 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589652 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.600769 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589655 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589657 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589660 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589662 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589665 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589667 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589670 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589672 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589675 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589677 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589680 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589683 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589686 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589688 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.589713 6492 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589749 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.601570 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589753 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589757 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589760 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589763 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589766 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589768 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589771 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589773 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589776 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589778 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589781 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589784 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589788 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589791 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589794 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589796 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589799 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589801 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589804 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.601992 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589806 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589809 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589811 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589815 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589819 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589821 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589824 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589827 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589829 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589832 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589834 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589837 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589839 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589842 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589847 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589850 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589853 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589855 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589858 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589861 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.602471 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589864 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589867 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589871 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589874 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589877 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589881 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589884 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589887 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589889 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589892 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589894 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589897 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589899 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589902 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589904 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589907 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589909 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589912 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589914 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.603124 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589917 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589919 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589922 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589924 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589927 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589929 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589931 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589935 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589938 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589941 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589943 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589946 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589948 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589951 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589953 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589956 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589958 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589961 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589964 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.603790 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589967 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589970 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589973 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589975 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589978 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589980 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589983 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589985 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589988 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589990 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589993 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589996 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.589998 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590001 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590003 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590006 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590008 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590011 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590013 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590016 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.604394 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590019 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590022 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590025 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590028 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590030 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590033 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590036 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.590038 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.594297 6492 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.594313 6492 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.594332 6492 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.594336 6492 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594426 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594431 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594435 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594437 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594440 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594443 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594446 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.604881 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594449 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594457 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594460 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594463 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594466 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594468 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594471 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594474 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594477 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594480 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594483 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594486 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594488 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594491 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594494 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594497 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594500 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594503 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594507 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.605337 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594511 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594515 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594518 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594521 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594523 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594526 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594529 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594532 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594534 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594551 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594555 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594558 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594561 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594564 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594566 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594575 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594578 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594581 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594584 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594586 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.605817 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594589 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594592 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594595 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594597 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594600 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594603 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594606 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594609 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594612 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594615 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594617 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594620 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594623 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594626 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594628 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594631 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594634 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594636 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594639 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.606312 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594642 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594644 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594647 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594651 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594655 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594658 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594661 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594664 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594666 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594675 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594678 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594681 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594684 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594687 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594689 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594692 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594695 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594698 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594701 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594704 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.606834 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594707 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594709 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594712 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594715 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594717 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594720 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594722 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594725 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594727 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594730 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594740 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594743 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594745 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594748 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594750 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594753 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594756 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594759 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594761 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594764 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.607362 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.594767 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.594796 6492 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595502 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595559 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595565 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595574 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595578 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595583 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595587 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595591 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595597 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595601 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595605 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595609 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595614 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.607934 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595618 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595623 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595627 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595636 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595640 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595657 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595662 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595666 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595670 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595675 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595679 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595683 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595688 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595692 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595696 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595704 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595708 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595713 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595717 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595721 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.608284 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595750 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595756 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595760 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595764 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595768 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595773 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595776 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595781 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595789 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595794 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595798 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595802 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595806 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595810 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595814 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595818 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595822 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595827 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595831 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595835 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.608765 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595839 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595847 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595852 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595857 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595861 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595865 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595869 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595873 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595877 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595881 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595885 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595889 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595893 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595929 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595933 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595937 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595967 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595973 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595978 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.609229 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595982 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595986 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595990 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595994 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.595998 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596003 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596007 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596015 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596019 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596023 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596027 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596031 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596035 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596039 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596043 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596047 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596053 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596084 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596089 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596109 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.609709 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596114 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596133 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596242 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596247 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596251 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596254 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596256 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596260 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596264 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596268 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596273 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596276 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596279 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596282 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.596319 6492 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596382 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:08.610304 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596388 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596391 6492 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596395 6492 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596398 6492 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596401 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596404 6492 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596406 6492 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596409 6492 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596412 6492 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596414 6492 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596417 6492 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596421 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596424 6492 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596427 6492 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596430 6492 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596432 6492 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596435 6492 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596438 6492 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596440 6492 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596443 6492 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:08.610921 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596446 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596448 6492 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596451 6492 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596454 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596457 6492 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596459 6492 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596462 6492 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596465 6492 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596467 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596470 6492 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596472 6492 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596475 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596477 6492 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596480 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596483 6492 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596485 6492 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596488 6492 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596490 6492 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596492 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:08.611524 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596495 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596498 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596506 6492 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596509 6492 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596512 6492 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596515 6492 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596518 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596522 6492 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596525 6492 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596528 6492 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596531 6492 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596534 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596552 6492 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596557 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596560 6492 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596563 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596566 6492 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596568 6492 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596571 6492 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:08.612046 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596573 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596576 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596578 6492 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596580 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596583 6492 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596585 6492 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596588 6492 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596591 6492 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596593 6492 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596596 6492 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596598 6492 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596601 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596603 6492 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596606 6492 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596608 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596612 6492 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596621 6492 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596624 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596626 6492 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:08.612603 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596629 6492 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596633 6492 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596636 6492 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596639 6492 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596642 6492 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596645 6492 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596648 6492 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596651 6492 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596653 6492 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596656 6492 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596658 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596661 6492 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596663 6492 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596666 6492 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596668 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596671 6492 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596674 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596676 6492 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:08.613112 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596679 6492 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596682 6492 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596685 6492 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596688 6492 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596692 6492 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596694 6492 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596697 6492 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596699 6492 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596702 6492 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:08.596704 6492 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.596784 6492 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.597760 6492 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.597811 6492 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.597980 6492 server.go:1014] "Starting client certificate rotation" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.598122 6492 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.598177 6492 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:56 +0000 UTC" deadline="2028-05-30 01:51:30.965888606 +0000 UTC" Oct 30 18:32:08.613596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.598253 6492 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13855h19m22.367658912s" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.598602 6492 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.599686 6492 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.601017 6492 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.602936 6492 log.go:25] "Validated CRI v1 image API" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.603724 6492 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:32:08.614014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.605990 6492 fs.go:135] Filesystem UUIDs: map[1f37733c-6da4-4440-beb6-301689f22fd0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e68c8ab5-5873-4257-acd4-114be8244df8:/dev/nvme0n1p3] Oct 30 18:32:08.614196 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.606013 6492 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2/userdata/shm major:0 minor:418 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd/userdata/shm major:0 minor:413 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae/userdata/shm major:0 minor:265 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96/userdata/shm major:0 minor:253 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a38dc92dc611e4aeb4cb4e5d9f215ae93ae14f15bfdbec4b676b6b5329146b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a38dc92dc611e4aeb4cb4e5d9f215ae93ae14f15bfdbec4b676b6b5329146b2/userdata/shm major:0 minor:415 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527/userdata/shm major:0 minor:260 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2/userdata/shm major:0 minor:371 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0/userdata/shm major:0 minor:410 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262/userdata/shm major:0 minor:304 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef2e7926dac7fd3af94a9537629f328ea6ee5e70364a782721ac01cbd193a3ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef2e7926dac7fd3af94a9537629f328ea6ee5e70364a782721ac01cbd193a3ef/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~projected/kube-api-access-g2pws:{mountpoint:/var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~projected/kube-api-access-g2pws major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~projected/kube-api-access-qfz84:{mountpoint:/var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~projected/kube-api-access-qfz84 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/195ffa97-91dc-4cad-802d-97027141794c/volumes/kubernetes.io~projected/kube-api-access-v7gmg:{mountpoint:/var/lib/kubelet/pods/195ffa97-91dc-4cad-802d-97027141794c/volumes/kubernetes.io~projected/kube-api-access-v7gmg major:0 minor:400 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/config-reloader/2 major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/prometheus/15 major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~empty-dir/config-out major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/kube-api-access-6kf9p:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/kube-api-access-6kf9p major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/tls-assets major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/config major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:365 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/web-config major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3423a0-15ee-4a92-a819-0507cf101d2c/volumes/kubernetes.io~projected/kube-api-access-m9z8f:{mountpoint:/var/lib/kubelet/pods/1f3423a0-15ee-4a92-a819-0507cf101d2c/volumes/kubernetes.io~projected/kube-api-access-m9z8f major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~projected/kube-api-access-66z2r:{mountpoint:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~projected/kube-api-access-66z2r major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3/volumes/kubernetes.io~projected/kube-api-access-qzc4f:{mountpoint:/var/lib/kubelet/pods/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3/volumes/kubernetes.io~projected/kube-api-access-qzc4f major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~projected/kube-api-access-zhhhq:{mountpoint:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~projected/kube-api-access-zhhhq major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~projected/kube-api-access-fh4g6:{mountpoint:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~projected/kube-api-access-fh4g6 major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~projected/kube-api-access-7g95f:{mountpoint:/var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~projected/kube-api-access-7g95f major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~secret/metrics-tls major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~projected/kube-api-access-mhf4n:{mountpoint:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~projected/kube-api-access-mhf4n major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~projected/kube-api-access-xt8jb:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~projected/kube-api-access-xt8jb major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~projected/kube-api-access-gf4mx:{mountpoint:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~projected/kube-api-access-gf4mx major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~projected/kube-api-access-d2rcc:{mountpoint:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~projected/kube-api-access-d2rcc major:0 minor:417 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:411 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:406 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3b81f81-6c7f-44cd-a040-e7488cfea341/volumes/kubernetes.io~projected/kube-api-access-lcmw8:{mountpoint:/var/lib/kubelet/pods/a3b81f81-6c7f-44cd-a040-e7488cfea341/volumes/kubernetes.io~projected/kube-api-access-lcmw8 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a56c3ed8-1972-402c-8195-b4dc842c69fb/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/a56c3ed8-1972-402c-8195-b4dc842c69fb/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:394 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~projected/kube-api-access-78st7:{mountpoint:/var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~projected/kube-api-access-78st7 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~secret/metrics-certs major:0 minor:395 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3b05773-655d-4d2e-a924-2a6746e2a1e6/volumes/kubernetes.io~projected/kube-api-access-nqnz9:{mountpoint:/var/lib/kubelet/pods/b3b05773-655d-4d2e-a924-2a6746e2a1e6/volumes/kubernetes.io~projected/kube-api-access-nqnz9 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba5fa788-50a7-47bb-bfb3-231034995cea/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ba5fa788-50a7-47bb-bfb3-231034995cea/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf7a8c1d-004b-423c-8d4c-27e2b6757aae/volumes/kubernetes.io~projected/kube-api-access-qj4vw:{mountpoint:/var/lib/kubelet/pods/cf7a8c1d-004b-423c-8d4c-27e2b6757aae/volumes/kubernetes.io~projected/kube-api-access-qj4vw major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~projected/kube-api-access-dbdj5:{mountpoint:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~projected/kube-api-access-dbdj5 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e025fb6d-2541-4243-a8d3-8fc4fc5f5940/volumes/kubernetes.io~projected/kube-api-access-2vjb5:{mountpoint:/var/lib/kubelet/pods/e025fb6d-2541-4243-a8d3-8fc4fc5f5940/volumes/kubernetes.io~projected/kube-api-access-2vjb5 major:0 minor:59 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/71d519f9128523c5037cdba0478a2aeec8958d5ef741fe24dc730925b57a6c34/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/1cfb73a62b7db1b6d6ae573b37845dc79e2bf1e6b6556307c4a080be43ef8ba4/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/aa87f6aaf2850a7a784051e35c888b9414a358988012273bbf45cfa23b2dc4c3/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/cba82a2d78d9cfb4abd6bdcd2ed4ad113a7392080ed887e272e0e68b472fcc93/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/2c9ea339d49cf4bf3b51faba815cfe33739b0bf873b4fc761a13d623c83bd7bd/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8a70ccfae89c9a650c00ea592cc27e4a1f531e726965820642beed3c2048449e/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1576a0446cc9b78bc9bba2ede761714c80f94d5699b0462d5f1aad64461615aa/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/55251ba683d407278008b94e88dfcbf50e78eafb6e21a6f53be5f590355ece29/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/22317afec10f2cc4f85b3853abe3c682b84d4dba10a3e12d7b5fb10c2d70544c/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/703ca978a0f1e23fa1cb29266ca5ac728fae43d9e971a9be0f2d4bcc1cc4deaf/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/1800a36c1d9c27b3db896c6a67b9376c63ce51b98f4919b9a357003ba82921d1/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/8206d1a0a2b88b568f066c3ee53463167f111e312827500b00eb9c4183b37e60/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/7f8051e829f5be5697a397311d1c4a6f9eb831746b07641e0837f7e995a8ec2f/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/39adf9f9e4f32aec55ebcf1266dcd638237a357abc84e9fa121d9def45bd1471/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/11047182c781364badac0cded660f96302a17539b8a91ca0fa94711fbe7cf5bc/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/297b24d859bad455108ab67a30910ce131c25bda6eb5164155ee2d2a8e4194b2/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/acde103a2d22ea23908cee17c7b3c86b4325280cadbf084a16f8a61c529fb543/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/93b4c99ccf4088e016b863dc043552b01352eb87f227c75719c1fccca1e74067/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/368e4684f21277d80ef091ce052a115126870d5b446594aa8efc960041df4f90/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/9ede2a4baca4d15df26923ca74f34ed61211c930c8231a37557acabf57757fa9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/9cbb0447a7d564b20b6c342a6afd79227268fd11d9d933b2b53ae22e34a5927c/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/e0a122299b7e54a4a0a40e1fa9dfc65e4145e91f2e1e975d3a1dbb36dde51fd2/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/2d4aa052c05d7a5fb5395974f6c6148caa3f4283565294417a97d81862f4592b/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/19b1a2434381286cb812a65f86599c55da8eaf2ded39ae225819717bc428994b/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/23b732ce655229115eb23feff8aed0a45559e83089151d011a8436c9a95c2010/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/3dd0e2a23a7ed3b2a06bff6a33da2f225553d8219b9604d1d42ba116069d2623/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/72409de0792f309f484a9e231a468a68841e79efd93f2c015f0e21c570cf9557/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/6402aeed4d74f05d22c0f48f4a715eb00753b8edaa298725bbcd829029513b7f/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/a641ba1b2bd2127f8a71f7d9cfcb75a48ae32407d2f27ba51994ba60eba7ce89/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/d32d833a8711fc8e78f22974258f133cdac118b7247f5af8204e5e344efee79b/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/e343275187162bd78c913e14033bd44d2085013987a356a2432dcf207e99eab1/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/694bf1d243fba4c8d540a3989c22e40356228e9e780b57bd443ac11752437217/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/c698848d5b1825a8786094b2be574b072db1cf877e5483449c13ce64d41ea0e4/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/2650b9a4bbb1ff938c32ca953aeacfa0b56b028290009b50abd2476a63d44438/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/0008cb50faeff327f57d09f455ed7247f0514514fb7b0e92429e533a8c9ac327/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/4fe02b7b013275be341016bd30dd1552b1475fe20ab5589768f6ef0421bb8973/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/d9401020b682be48289d9b03678dddab51b3480b7390a3a988cfc83cb33c7bcd/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/ba7f0e158eeb0ddda653fe21a22b097f5a724946995c040ea21dd990e022d56d/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/23a9a2974c9d4888c74b0f80672774c7c514b86a8212b709aea703a0d6a372a3/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/dc0a7e59260e9968ebd34eb4a84f86009532ff1d879d9f9e4a1dfeeba07470de/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/e9ab7313ab57d84bb922769d8411d90d7e75a5ccfc49f148b187f9d7370e98b4/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/e45e59f1d54ab48f172584ca11d6990fd32ebcc844ea3e3cb81d790448a70f75/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/6d7ec00617ff515ca8b436b6754e3b1f25c6842d3fe1408589a43e4247991ce6/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/a66f57805e457fa176441bd7c75a3cc93681a048a53b17c6ef878a8aa5cc10a5/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/0f065574a105ce0278b484a412fb3a12e813548a5d90c29276d606fbd4a2264f/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/e1d97f59a3251110fb27aa8b573e9dbb1204c30f49bc15bb44da4d97d590e7f9/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/f039884eb586a346e1bdf98882378f23d4bbfce22a2f64a7d67f7b387382c37f/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/6ba29635b92ff658a631bb793065928ab5aca35a286eb821cc456b31c03d47ae/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/57d25cb58dbae458257f16041a79dd90858a11244ebd7c88190b9dbb1ba63530/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/86232eb42f3c26afb9e82ea0cb781e12cf11fc93342d0d4999f218c3f72d1397/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/eabf7e1f49be740e0fd15566c0da8de985bd035db6b5882df6136dd2d78bc7ef/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-383:{mountpoint:/var/lib/containers/storage/overlay/b18602b09aaf372a8691e1dd9a117f7c61e8165d68256604ff9f177b39f6eb03/merged major:0 minor:383 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/db60a3683eca696969ce6bf3af505da23b8cc6bf8e499133f5c5a422ff816544/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/df99a5d3be93f74b9338eeae875ef54e41955f71436c3aabed2a7b7ade8c3b84/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/c6833d2b34fc9d5c10f8ab3567bf1c2a5ff88bfefc9e7178e137cb7e0638b8e2/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/c5d626b98c4ff26722b579963605073bb5624d53108c8a19a583065341b5758d/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/00807cf946bacda683f551d6b8eb99e86a196a936809acb83aa6d4259497fabf/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/790b7721e67d190fa8d338e634caa708b93a38468cae95b66d299022f77ab25d/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/5a8181a2bcb539ddedc6c2a0dc979147cc6bcc2110c38e8319ce79a53f4f7fc5/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-428:{mountpoint:/var/lib/containers/storage/overlay/a50bec06b67bbc1811e0b09559b7269cdbbbb94294d21c5fd16b5c3fb9d6bf63/merged major:0 minor:428 fsType:overlay blockSize:0} overlay_0-430:{mountpoint:/var/lib/containers/storage/overlay/6c61bbabace47f2e942775d27617c6de31c7fd2b7b66f0d9278e9fb4c32c022f/merged major:0 minor:430 fsType:overlay blockSize:0} overlay_0-436:{mountpoint:/var/lib/containers/storage/overlay/26aab725d22dbf53424cc02b45ffb4e155dff0f6d66c0d1679e954c1f3649285/merged major:0 minor:436 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/58c2e6a4d7b5cb9e2761054723ee9ec1375a364848bc567828a2bd8df0d64c58/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-447:{mountpoint:/var/lib/containers/storage/overlay/0f024dbf85ac67a81ea4c88f2c06634766e8df3095810333229157d2b127784d/merged major:0 minor:447 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/e285b52b472f91cc7d24710b4662bcc243bf33a6787ea00215607623e746a5a8/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/31c62976ef36ffd7c8f7aa46054ae7c6a08ebf8f71b801e739920a4460b4c5de/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-71:{mountpoint:/var/lib/containers/storage/overlay/ca282e2e7beb76ac457662922d3f5d41dc29c82bd0fc5a8b8ba0f376b1e56016/merged major:0 minor:71 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/ccad49f8bed78b8bd77c1a67860c11bec042671deb227bcae48fe7acf325d184/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/f8bcefd6e568adec93ffbcf773f362e0516ad9d456ce0f249e605ae301b1a539/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/9d039e9889352ea0e1fcb082fb2eff15a05fabb5f25b361386d82f8ab4aee02b/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/f0ac3cae7356dc826c4aee93f17fc2933c88415b662cb445faaf9c3d7d80a92a/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/a3bab9aa2da3de47f788061707158072db9c964c364b76c1917e2a55c0c0cf43/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/cb4d185934152be72e3869c5975700cff74aad8bd95a3ee7a278a472f31aaa98/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/3dd0fccfdde2e90044c3b11e9a4a4cbf3ad8ba3476be076abc919c3734fb9865/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:32:08.616497 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.616092 6492 manager.go:217] Machine: {Timestamp:2026-10-30 18:32:08.614813619 +0000 UTC m=+0.059204607 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3214626 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239e5d8c30c91a2b2058514dcb09a4 SystemUUID:ec239e5d-8c30-c91a-2b20-58514dcb09a4 BootID:5caf9318-4e2a-48fe-be12-b40c781dfe64 Filesystems:[{Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/kube-api-access-6kf9p DeviceMajor:0 DeviceMinor:370 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/670b6d8e8eca9a0f56cb147402b77bff8026b88b0d5795b853075495077d1e6f/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:364 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e025fb6d-2541-4243-a8d3-8fc4fc5f5940/volumes/kubernetes.io~projected/kube-api-access-2vjb5 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3/volumes/kubernetes.io~projected/kube-api-access-qzc4f DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c38ab3c2042faa2b6ce9a8778529b02089e9e6ddf74042d02ce1c63f540fcf2/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:326 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:363 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eac517eefb4ebde8526f6370021591fca5ca242e9159e608224bc6b08eae71f2/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:251 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:262 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96/userdata/shm DeviceMajor:0 DeviceMinor:253 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a3b81f81-6c7f-44cd-a040-e7488cfea341/volumes/kubernetes.io~projected/kube-api-access-lcmw8 DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:246 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~projected/kube-api-access-xt8jb DeviceMajor:0 DeviceMinor:303 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-383 DeviceMajor:0 DeviceMinor:383 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~projected/kube-api-access-gf4mx DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:355 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:366 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:406 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes/kubernetes.io~projected/kube-api-access-mhf4n DeviceMajor:0 DeviceMinor:252 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f82065d11657fd7d90b627fc46074259cf83bc388a66fea936ea94c23268e29/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/24cc86a6-a954-4a74-a70b-63e31860233c/volumes/kubernetes.io~projected/kube-api-access-66z2r DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:365 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:369 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:411 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/502b8dd3-61d1-4a04-883e-35cf6460a661/volumes/kubernetes.io~projected/kube-api-access-7g95f DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0c7fe3f2-f764-4843-8732-f5434a5f82cc/volumes/kubernetes.io~projected/kube-api-access-g2pws DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/195ffa97-91dc-4cad-802d-97027141794c/volumes/kubernetes.io~projected/kube-api-access-v7gmg DeviceMajor:0 DeviceMinor:400 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11357dc0-5895-4a3d-9969-209df38fa006/volumes/kubernetes.io~projected/kube-api-access-qfz84 DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:258 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:302 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2/userdata/shm DeviceMajor:0 DeviceMinor:371 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a38dc92dc611e4aeb4cb4e5d9f215ae93ae14f15bfdbec4b676b6b5329146b2/userdata/shm DeviceMajor:0 DeviceMinor:415 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ba5fa788-50a7-47bb-bfb3-231034995cea/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b3b05773-655d-4d2e-a924-2a6746e2a1e6/volumes/kubernetes.io~projected/kube-api-access-nqnz9 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6acc006346d2daca721fcf7ba44a1215f66fbc2d0647385996c28ae18465fa2f/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:294 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:298 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:364 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5641ee499fcac5546834d8f8d6bc8ec817fb7f6f1827ad54293c40ecceb925e3/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:257 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4bd5c23b-225b-4eff-8b03-77fe7c04e56c/volumes/kubernetes.io~projected/kube-api-access-fh4g6 DeviceMajor:0 DeviceMinor:259 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/16e0e4588a97998b9090021c6827a32c922214a04cff2c0d4248b0672d3217cd/userdata/shm DeviceMajor:0 DeviceMinor:413 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-430 DeviceMajor:0 DeviceMinor:430 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f0ec6281dd4e56a1da295f2a704e7655b34418cd1b274a666ea6b4bd57831c9e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:263 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/3dc7152c-07ad-466c-aed1-1e289c384fde/volumes/kubernetes.io~projected/kube-api-access-zhhhq DeviceMajor:0 DeviceMinor:264 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-428 DeviceMajor:0 DeviceMinor:428 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf7a8c1d-004b-423c-8d4c-27e2b6757aae/volumes/kubernetes.io~projected/kube-api-access-qj4vw DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f82e372a72a0f337baf6e941c1fb956b59030529a38622d326d9c51796f0527/userdata/shm DeviceMajor:0 DeviceMinor:260 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3423a0-15ee-4a92-a819-0507cf101d2c/volumes/kubernetes.io~projected/kube-api-access-m9z8f DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d299a5b87dc205ed717eaa27e2e02566b0502b72a3843a31b35991ebd7923ae/userdata/shm DeviceMajor:0 DeviceMinor:265 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a56c3ed8-1972-402c-8195-b4dc842c69fb/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:394 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:395 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/114ec8143f2f74ded69152c14de7dee985d9533185bc207774afa89c0b114f8b/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-447 DeviceMajor:0 DeviceMinor:447 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/560ff43249e545f5b64bb952f03a07397416855d23cb95249e28db37b87c4adb/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:301 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:328 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bda7b7196f240ab5284eb766f6e266ba4eb86c336c9e3eae8e459ca19728abe0/userdata/shm DeviceMajor:0 DeviceMinor:410 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes/kubernetes.io~projected/kube-api-access-d2rcc DeviceMajor:0 DeviceMinor:417 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-71 DeviceMajor:0 DeviceMinor:71 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:368 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2/userdata/shm DeviceMajor:0 DeviceMinor:418 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef2e7926dac7fd3af94a9537629f328ea6ee5e70364a782721ac01cbd193a3ef/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:299 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:300 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b2780b90-9d95-44d8-bc10-8294255f08a5/volumes/kubernetes.io~projected/kube-api-access-78st7 DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dbee4f62-3043-46ce-821a-b77f3746f83a/volumes/kubernetes.io~projected/kube-api-access-dbdj5 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:364 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e11954408a8cd36b5d5f8df1dabacd6cee4c58a688f96dccd528ab762a1b9fe/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d713cda008a7be3f753149808ec023dddc0177e77cf42d823c14724bb8aba262/userdata/shm DeviceMajor:0 DeviceMinor:304 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-436 DeviceMajor:0 DeviceMinor:436 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/852071ed-f1b5-4b6e-8ef5-1deb88d86855/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:367 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:362 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f816496a4c192fc3a2ea7b56e01c64573278d21c341291912f0ca729be4e8d5d/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:364 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:000f16ddec881ac MacAddress:fa:85:89:62:68:44 Speed:10000 Mtu:8901} {Name:16e0e4588a97998 MacAddress:b6:88:8f:0b:d0:dc Speed:10000 Mtu:8901} {Name:2f82065d11657fd MacAddress:5e:da:90:0e:96:26 Speed:10000 Mtu:8901} {Name:3eceff6c5901e47 MacAddress:56:16:26:6f:da:14 Speed:10000 Mtu:8901} {Name:560ff43249e545f MacAddress:82:5c:35:6d:cc:76 Speed:10000 Mtu:8901} {Name:8a38dc92dc611e4 MacAddress:1a:4a:a9:7c:1b:d6 Speed:10000 Mtu:8901} {Name:8f82e372a72a0f3 MacAddress:0a:f9:dc:2f:67:af Speed:10000 Mtu:8901} {Name:a93ec0ec9c8a627 MacAddress:c2:36:6e:22:19:f5 Speed:10000 Mtu:8901} {Name:bda7b7196f240ab MacAddress:a6:1a:2b:70:54:34 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d1:62:cf:95 Speed:0 Mtu:9001} {Name:br-int MacAddress:9a:f0:5c:b7:df:63 Speed:0 Mtu:8901} {Name:d713cda008a7be3 MacAddress:26:8b:60:50:f8:23 Speed:10000 Mtu:8901} {Name:ef2e7926dac7fd3 MacAddress:22:1f:14:66:c4:73 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:d1:62:cf:95 Speed:0 Mtu:9001} {Name:f816496a4c192fc MacAddress:de:e2:d0:08:90:4d Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:b2:b2:42:ce:3b:9a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:a6:01:a1:63:4c:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:32:08.616497 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.616499 6492 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:32:08.616830 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.616570 6492 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:32:08.616890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.616864 6492 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:32:08.617023 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.616891 6492 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:32:08.617064 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617025 6492 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:32:08.617064 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617033 6492 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:32:08.617064 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617043 6492 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:32:08.617064 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617063 6492 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:32:08.617171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617168 6492 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:32:08.617336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617326 6492 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:32:08.617451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617442 6492 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:32:08.617490 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617483 6492 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:32:08.617522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617498 6492 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:32:08.617522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617508 6492 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:32:08.617522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617518 6492 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:32:08.617618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.617526 6492 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:32:08.618260 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.618229 6492 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:32:08.618260 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.618260 6492 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:32:08.618780 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.618764 6492 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:32:08.621335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621322 6492 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:32:08.621384 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621335 6492 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:32:08.621437 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621428 6492 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:32:08.621638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621626 6492 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:32:08.621675 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621655 6492 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:32:08.621745 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621736 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:32:08.621793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621750 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:32:08.621793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621755 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:32:08.621793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621782 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:32:08.621793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621790 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621799 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621806 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621814 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621824 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621831 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621850 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621863 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621891 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:32:08.621935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.621896 6492 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:32:08.625253 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.625239 6492 server.go:1263] "Started kubelet" Oct 30 18:32:08.625337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.625282 6492 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:32:08.625441 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.625379 6492 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:32:08.625856 ip-10-0-136-105 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:32:08.626094 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.626069 6492 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:32:08.626448 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.626423 6492 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:32:08.626848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.626830 6492 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:32:08.631734 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.631715 6492 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:32:08.632143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632035 6492 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632486 6492 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:58 +0000 UTC" deadline="2028-07-29 01:27:14.632631922 +0000 UTC" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632532 6492 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15294h55m6.000103907s" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632726 6492 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632739 6492 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.632812 6492 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.632795 6492 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-105.ec2.internal\" not found" Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.633127 6492 factory.go:55] Registering systemd factory Oct 30 18:32:08.633571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.633146 6492 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:32:08.634421 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634400 6492 factory.go:153] Registering CRI-O factory Oct 30 18:32:08.634421 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634421 6492 factory.go:223] Registration of the crio container factory successfully Oct 30 18:32:08.634595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634476 6492 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:32:08.634595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634494 6492 factory.go:103] Registering Raw factory Oct 30 18:32:08.634595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634508 6492 manager.go:1199] Started watching for new ooms in manager Oct 30 18:32:08.634890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.634876 6492 manager.go:319] Starting recovery of all containers Oct 30 18:32:08.635907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635870 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:08.635967 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635912 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p" seLinuxMountContext="" Oct 30 18:32:08.635967 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635929 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3" volumeName="kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f" seLinuxMountContext="" Oct 30 18:32:08.635967 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635941 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:08.635967 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635955 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bd5c23b-225b-4eff-8b03-77fe7c04e56c" volumeName="kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635967 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635979 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.635992 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="195ffa97-91dc-4cad-802d-97027141794c" volumeName="kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636009 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3" volumeName="kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636021 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636033 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636059 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636072 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc" seLinuxMountContext="" Oct 30 18:32:08.636091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636083 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2780b90-9d95-44d8-bc10-8294255f08a5" volumeName="kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636094 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b05773-655d-4d2e-a924-2a6746e2a1e6" volumeName="kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636102 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c7fe3f2-f764-4843-8732-f5434a5f82cc" volumeName="kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636109 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636117 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="502b8dd3-61d1-4a04-883e-35cf6460a661" volumeName="kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636124 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e025fb6d-2541-4243-a8d3-8fc4fc5f5940" volumeName="kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636131 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c7fe3f2-f764-4843-8732-f5434a5f82cc" volumeName="kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636139 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636147 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3423a0-15ee-4a92-a819-0507cf101d2c" volumeName="kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636154 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bd5c23b-225b-4eff-8b03-77fe7c04e56c" volumeName="kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636161 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636169 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636176 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3b81f81-6c7f-44cd-a040-e7488cfea341" volumeName="kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636185 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b05773-655d-4d2e-a924-2a6746e2a1e6" volumeName="kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636206 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11357dc0-5895-4a3d-9969-209df38fa006" volumeName="kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636236 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636248 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b05773-655d-4d2e-a924-2a6746e2a1e6" volumeName="kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636257 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbee4f62-3043-46ce-821a-b77f3746f83a" volumeName="kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636265 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636274 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636282 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636289 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636303 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba5fa788-50a7-47bb-bfb3-231034995cea" volumeName="kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca" seLinuxMountContext="" Oct 30 18:32:08.636319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636325 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636332 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636341 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636348 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3b81f81-6c7f-44cd-a040-e7488cfea341" volumeName="kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636361 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636368 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24cc86a6-a954-4a74-a70b-63e31860233c" volumeName="kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636375 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24cc86a6-a954-4a74-a70b-63e31860233c" volumeName="kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636382 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636390 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636397 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbee4f62-3043-46ce-821a-b77f3746f83a" volumeName="kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636404 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbee4f62-3043-46ce-821a-b77f3746f83a" volumeName="kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636412 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636425 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636432 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636439 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636446 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636455 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c7fe3f2-f764-4843-8732-f5434a5f82cc" volumeName="kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636462 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636469 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636477 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636504 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3b81f81-6c7f-44cd-a040-e7488cfea341" volumeName="kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636514 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf7a8c1d-004b-423c-8d4c-27e2b6757aae" volumeName="kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636521 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636530 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636558 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636570 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636582 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba5fa788-50a7-47bb-bfb3-231034995cea" volumeName="kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636593 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbee4f62-3043-46ce-821a-b77f3746f83a" volumeName="kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636615 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636626 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636637 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636650 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24cc86a6-a954-4a74-a70b-63e31860233c" volumeName="kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636664 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636678 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636692 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636703 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636717 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636732 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b05773-655d-4d2e-a924-2a6746e2a1e6" volumeName="kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636747 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbee4f62-3043-46ce-821a-b77f3746f83a" volumeName="kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636770 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11357dc0-5895-4a3d-9969-209df38fa006" volumeName="kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636784 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636796 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dc7152c-07ad-466c-aed1-1e289c384fde" volumeName="kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636807 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="502b8dd3-61d1-4a04-883e-35cf6460a661" volumeName="kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636814 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="502b8dd3-61d1-4a04-883e-35cf6460a661" volumeName="kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636840 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="502b8dd3-61d1-4a04-883e-35cf6460a661" volumeName="kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636852 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636865 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636897 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636910 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636924 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636937 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bd5c23b-225b-4eff-8b03-77fe7c04e56c" volumeName="kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636952 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636964 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b" volumeName="kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636977 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.636989 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852071ed-f1b5-4b6e-8ef5-1deb88d86855" volumeName="kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637000 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19921861-8bf5-4a85-b934-56a1b07cf4ae" volumeName="kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637021 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bd5c23b-225b-4eff-8b03-77fe7c04e56c" volumeName="kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637033 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6487588c-2589-404f-8711-3ccf6a999ebb" volumeName="kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637044 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a9150d1-1c80-4155-9b40-64793e254389" volumeName="kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config" seLinuxMountContext="" Oct 30 18:32:08.637058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637067 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a56c3ed8-1972-402c-8195-b4dc842c69fb" volumeName="kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret" seLinuxMountContext="" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637096 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2780b90-9d95-44d8-bc10-8294255f08a5" volumeName="kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7" seLinuxMountContext="" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637107 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf7a8c1d-004b-423c-8d4c-27e2b6757aae" volumeName="kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca" seLinuxMountContext="" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637128 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e025fb6d-2541-4243-a8d3-8fc4fc5f5940" volumeName="kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir" seLinuxMountContext="" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637140 6492 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c7fe3f2-f764-4843-8732-f5434a5f82cc" volumeName="kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637151 6492 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:32:08.639063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.637159 6492 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:32:08.656133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.656098 6492 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:32:08.657431 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.657398 6492 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:32:08.657578 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.657428 6492 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:32:08.657578 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.657461 6492 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:32:08.657578 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.657561 6492 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:32:08.663068 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.663028 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:32:08.663654 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.663633 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:32:08.675760 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.675732 6492 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:32:08.677923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.677897 6492 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:08.678161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.678135 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" exitCode=0 Oct 30 18:32:08.678493 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.678466 6492 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:08.685977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685950 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="c4cdc7708e720ffc60ed5079e16ef74181d79ef32e241735f56272e6c245049e" exitCode=0 Oct 30 18:32:08.685977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685973 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="379e0d8c424c85fedaa4436071f2d842fadbabeeed3b789c7cd12aa103ad8845" exitCode=0 Oct 30 18:32:08.685977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685980 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="499d97ae9783f008d05ee7812083b91d36c9aa86475da55437cae355477e668d" exitCode=0 Oct 30 18:32:08.686171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685985 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="d50e7886aa01de211502966cf9ac9d2ccf29d411376ecdcd08f29ed36319addf" exitCode=0 Oct 30 18:32:08.686171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685990 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="19d9d7af7f4ce6a231d1f9f629c88ec5ae183b665415333a96641dfe33f5fff8" exitCode=0 Oct 30 18:32:08.686171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.685995 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3b05773-655d-4d2e-a924-2a6746e2a1e6" containerID="fd1828088bf9e02ea21f391dfde3a5e5976a411648a914e8c09aec143e0679b8" exitCode=0 Oct 30 18:32:08.687798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.687774 6492 generic.go:356] "Generic (PLEG): container finished" podID="8af086efb620e723a004b0f8b9bdafb8" containerID="49c728f2745051b6f69f8131952a098d260ade953080b68807a7c710388c90a5" exitCode=0 Oct 30 18:32:08.688102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.688083 6492 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:08.696232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.696204 6492 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:08.702978 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.702955 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:32:08.703296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.703268 6492 generic.go:356] "Generic (PLEG): container finished" podID="dbee4f62-3043-46ce-821a-b77f3746f83a" containerID="ac3e829195363f8fc97a94e0071d978d6f10bd577854aa276b3d57cc8a141d99" exitCode=1 Oct 30 18:32:08.721131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.721098 6492 generic.go:356] "Generic (PLEG): container finished" podID="3dc7152c-07ad-466c-aed1-1e289c384fde" containerID="e09b837868e685eede2e3696410f7f24c651bf1fde47486cf1fbf49b749a6f16" exitCode=0 Oct 30 18:32:08.743198 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.743168 6492 manager.go:324] Recovery completed Oct 30 18:32:08.758405 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.758320 6492 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:32:08.768524 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768500 6492 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:32:08.768524 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768518 6492 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:32:08.768720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768553 6492 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:32:08.768720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768692 6492 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:32:08.768720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768705 6492 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:32:08.768833 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768723 6492 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:32:08.768833 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768738 6492 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:32:08.768833 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768743 6492 policy_none.go:50] "Start" Oct 30 18:32:08.768833 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768752 6492 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:32:08.768833 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.768763 6492 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:32:08.769822 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.769805 6492 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:32:08.769859 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.769824 6492 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:32:08.769859 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.769831 6492 policy_none.go:44] "Start" Oct 30 18:32:08.774405 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.774377 6492 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:32:08.776968 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.776954 6492 manager.go:341] "Starting Device Plugin manager" Oct 30 18:32:08.777510 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.776981 6492 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:32:08.777620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.777513 6492 server.go:87] "Starting device plugin registration server" Oct 30 18:32:08.777813 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.777802 6492 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:32:08.777844 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.777814 6492 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:32:08.777942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.777918 6492 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:32:08.778071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.778036 6492 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:32:08.778071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.778080 6492 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:32:08.778187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.778088 6492 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:32:08.778187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.778119 6492 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:32:08.778068605Z","UUID":"f54ce555-ebb3-4d88-b40e-debfdbed6fd4","Handler":null,"Name":"","Endpoint":""} Oct 30 18:32:08.779722 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.779702 6492 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:32:08.779820 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.779743 6492 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:32:08.779975 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.779952 6492 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:32:08.878210 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.878172 6492 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:32:08.880346 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.880323 6492 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:32:08.880452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.880360 6492 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:32:08.880452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.880373 6492 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-105.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:32:08.880452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.880418 6492 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.914736 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.914705 6492 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.914877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.914805 6492 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.958876 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.958798 6492 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal"] Oct 30 18:32:08.959516 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959476 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerStarted","Data":"41e3cd50b32e21b93774c19413646bdf190f9714b3937103ff1c6406969ed37c"} Oct 30 18:32:08.959636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959528 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerDied","Data":"49c728f2745051b6f69f8131952a098d260ade953080b68807a7c710388c90a5"} Oct 30 18:32:08.959636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959567 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" event={"ID":"8af086efb620e723a004b0f8b9bdafb8","Type":"ContainerStarted","Data":"fd3b3fc3359123e04360219c8b2ac4a1bec12894c16a5828acd33e9ebebae978"} Oct 30 18:32:08.959636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959609 6492 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.959753 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959638 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" event={"ID":"c2a961c4fb7c61fa12bec5d71405aa03","Type":"ContainerStarted","Data":"7b273e1bbb5291102816a3fb98b6f3fb375578f274e9281b1e3aa2021a424c89"} Oct 30 18:32:08.959753 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959653 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" event={"ID":"c2a961c4fb7c61fa12bec5d71405aa03","Type":"ContainerStarted","Data":"c44633b0d4a0d454af4af35ea12a86a72158c62a6c5a6b2e44dd1138bf057d89"} Oct 30 18:32:08.959753 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.959686 6492 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.987339 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.987320 6492 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:08.987385 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.987370 6492 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:08.987422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:08.987320 6492 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:08.987476 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:08.987439 6492 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.078891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.078803 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.078891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.078852 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.078891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.078870 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179493 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179424 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179493 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179497 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179725 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179528 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179725 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179535 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179725 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179601 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c2a961c4fb7c61fa12bec5d71405aa03-config\") pod \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" (UID: \"c2a961c4fb7c61fa12bec5d71405aa03\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.179725 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.179607 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8af086efb620e723a004b0f8b9bdafb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" (UID: \"8af086efb620e723a004b0f8b9bdafb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.618164 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.618124 6492 apiserver.go:52] "Watching apiserver" Oct 30 18:32:09.664216 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.664183 6492 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:32:09.664831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.664805 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb","openshift-dns/dns-default-c8dqv","openshift-network-operator/iptables-alerter-5xlpr","kube-system/global-pull-secret-syncer-t6kkd","openshift-ovn-kubernetes/ovnkube-node-6v4xj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal","openshift-multus/multus-additional-cni-plugins-bpfcd","kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal","openshift-ingress-canary/ingress-canary-7fvzb","kube-system/konnectivity-agent-rl74h","openshift-console/console-7cbcd65b88-x42qd","openshift-image-registry/node-ca-fx675","openshift-monitoring/metrics-server-7969db6d87-blhz9","openshift-monitoring/node-exporter-24l9n","openshift-multus/network-metrics-daemon-km5cq","openshift-multus/multus-vsk5n","openshift-cluster-node-tuning-operator/tuned-mlq92","openshift-dns/node-resolver-9r555","openshift-monitoring/prometheus-k8s-0","openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7","openshift-network-diagnostics/network-check-target-x7rq7","openshift-insights/insights-runtime-extractor-w9swt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44","openshift-console/console-6ff767d568-st4n4"] Oct 30 18:32:09.684284 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684251 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:32:09.684468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684374 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:32:09.684468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684392 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.684468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684392 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.684468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684396 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:32:09.684468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684440 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684494 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684514 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684524 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684553 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684561 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684601 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684600 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684628 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:32:09.684705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.684664 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.694832 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.694812 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:32:09.695393 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.695379 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:32:09.696857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.696843 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.696973 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.696961 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:32:09.697010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.696996 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.699242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.699222 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:32:09.699285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.699251 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:32:09.699322 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.699300 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:32:09.724904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.724882 6492 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.725040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.725017 6492 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.735430 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.735407 6492 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:32:09.746069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.746046 6492 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:09.746217 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:09.746101 6492 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.746343 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.746325 6492 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:09.746394 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:09.746363 6492 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-105.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-105.ec2.internal" Oct 30 18:32:09.782196 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782150 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.782196 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782196 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782226 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782268 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782291 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782312 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782334 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.782457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782436 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782469 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782487 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz84\" (UniqueName: \"kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782508 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782525 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782556 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782591 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.782738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782714 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782747 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782835 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782895 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782980 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.782982 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.783022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783015 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783038 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783056 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783058 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a56c3ed8-1972-402c-8195-b4dc842c69fb-original-pull-secret\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783082 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783115 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78st7\" (UniqueName: \"kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783148 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783218 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.783254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783250 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhhhq\" (UniqueName: \"kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783278 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783307 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783350 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783371 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783383 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783411 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783499 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783553 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.783598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783594 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783643 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783677 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783751 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783768 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783865 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.783965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783909 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.784199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.783995 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.784199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784028 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.784199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784082 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.784199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784120 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.784372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784243 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9z8f\" (UniqueName: \"kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.784372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784303 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.784372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784336 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.784372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784362 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.784569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784384 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.784569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784409 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.784569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784448 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbdj5\" (UniqueName: \"kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.784569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784482 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.784701 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784676 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.784737 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784716 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.784773 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784743 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.784773 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784767 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.784863 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784796 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.784963 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784941 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.785014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.784974 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:09.785046 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785015 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzc4f\" (UniqueName: \"kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.785110 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785089 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.785159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785122 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-cni-binary-copy\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.785159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785133 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.785240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785222 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785255 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785226 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-textfile\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785284 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2780b90-9d95-44d8-bc10-8294255f08a5-metrics-certs\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785295 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785316 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.785321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785301 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785368 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785368 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785421 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785427 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785481 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785500 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785516 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785563 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.785576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785575 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785591 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt8jb\" (UniqueName: \"kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785613 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785633 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785630 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785668 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785687 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785704 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785736 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785765 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785791 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785818 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2pws\" (UniqueName: \"kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785864 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785881 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785892 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:32:09.785979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785919 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj4vw\" (UniqueName: \"kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.785994 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh4g6\" (UniqueName: \"kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786007 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786018 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786064 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-iptables-alerter-script\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786070 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786097 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786115 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7lgxg\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786122 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786149 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786177 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786186 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786190 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-etc-tuned\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786213 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnz9\" (UniqueName: \"kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786222 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786232 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786009 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786284 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bqjtk\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786295 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786306 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786360 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5se1021gu1ucd\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786368 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786401 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786433 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8lqh3egd5j83h\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786442 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r2wdj\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786072 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twcvb\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786433 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786500 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786473 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786535 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786455 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786474 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786403 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786635 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786681 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786685 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.786690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786714 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786724 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786725 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786740 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786766 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786780 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s77r6\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786792 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786812 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786819 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786870 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786883 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786894 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786918 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786951 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786964 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.786768 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787012 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787035 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787065 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787112 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787132 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vz2m7\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787138 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787166 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787179 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787187 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787289 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787305 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7fe3f2-f764-4843-8732-f5434a5f82cc-data-volume\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787358 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787369 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502b8dd3-61d1-4a04-883e-35cf6460a661-tmp-dir\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787428 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7fe3f2-f764-4843-8732-f5434a5f82cc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787431 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/852071ed-f1b5-4b6e-8ef5-1deb88d86855-audit-log\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787439 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787478 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11357dc0-5895-4a3d-9969-209df38fa006-cert\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787498 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787531 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787600 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba5fa788-50a7-47bb-bfb3-231034995cea-agent-certs\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787654 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787687 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787730 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787734 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbee4f62-3043-46ce-821a-b77f3746f83a-ovn-node-metrics-cert\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787761 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787851 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787873 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66z2r\" (UniqueName: \"kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.787941 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-serviceca\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788015 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g95f\" (UniqueName: \"kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788075 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788109 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788174 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788207 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6harcg9ua62ce\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788214 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788238 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788253 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788253 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-tmp-dir\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788267 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788280 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788294 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788312 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:32:09.788271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788316 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788312 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788312 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788357 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788387 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788431 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788464 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788492 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788586 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788607 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788625 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788647 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788667 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788684 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788706 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788711 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788730 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788760 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788785 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788805 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-client-certs\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788813 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788840 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cc86a6-a954-4a74-a70b-63e31860233c-tmp\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788841 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-secret-metrics-server-tls\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788787 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788862 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788897 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788906 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788929 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788944 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788969 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-tmp\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788970 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.788990 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789018 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789153 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789195 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789234 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789262 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789291 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789372 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789427 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789455 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789468 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789567 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789586 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789622 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjb5\" (UniqueName: \"kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789676 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-script-lib\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789736 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4mx\" (UniqueName: \"kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789755 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789846 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789900 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-grpc-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789950 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.789976 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790054 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790088 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790140 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790155 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790171 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790190 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.790371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790348 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcmw8\" (UniqueName: \"kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.791998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790436 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.791998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790454 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.791998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.790466 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.792891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.792867 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.793009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.792901 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-tls\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.793009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.792923 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.793009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.792970 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.793135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.793029 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.793135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.793049 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.793135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.793115 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794371 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794382 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794438 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794447 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794495 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794511 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794513 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794526 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/852071ed-f1b5-4b6e-8ef5-1deb88d86855-metrics-server-audit-profiles\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794534 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794529 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794579 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794615 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794625 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794635 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794653 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794668 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794679 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794693 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794694 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-env-overrides\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794713 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794719 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794729 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794752 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794780 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794798 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794852 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794852 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-secret-thanos-querier-tls\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794859 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.794942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.794915 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/502b8dd3-61d1-4a04-883e-35cf6460a661-metrics-tls\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.795975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.795769 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:32:09.795975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.795814 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:32:09.796293 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.796239 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.796293 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.796252 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.796293 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.796269 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.796466 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.796312 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.796678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.796662 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/502b8dd3-61d1-4a04-883e-35cf6460a661-config-volume\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.797566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.797526 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/852071ed-f1b5-4b6e-8ef5-1deb88d86855-client-ca-bundle\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.797566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.797559 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.797816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.797794 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 18:32:09.797881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.797815 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-metrics-client-ca\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.797881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.797842 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.798059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.798035 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.798367 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.798347 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:32:09.798434 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.798409 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:32:09.799106 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.799086 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dbee4f62-3043-46ce-821a-b77f3746f83a-ovnkube-config\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.799178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.799132 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.799178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.799147 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.799261 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.799213 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba5fa788-50a7-47bb-bfb3-231034995cea-konnectivity-ca\") pod \"konnectivity-agent-rl74h\" (UID: \"ba5fa788-50a7-47bb-bfb3-231034995cea\") " pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:32:09.800299 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.800285 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-metrics-client-ca\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.802859 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.802840 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.802936 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.802905 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.802936 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.802909 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.803656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.803640 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.803768 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.803752 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.804033 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.804019 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfz84\" (UniqueName: \"kubernetes.io/projected/11357dc0-5895-4a3d-9969-209df38fa006-kube-api-access-qfz84\") pod \"ingress-canary-7fvzb\" (UID: \"11357dc0-5895-4a3d-9969-209df38fa006\") " pod="openshift-ingress-canary/ingress-canary-7fvzb" Oct 30 18:32:09.804165 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.804152 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.804813 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.804799 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-accelerators-collector-config\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.804849 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.804819 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-daemon-config\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.806918 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.806901 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78st7\" (UniqueName: \"kubernetes.io/projected/b2780b90-9d95-44d8-bc10-8294255f08a5-kube-api-access-78st7\") pod \"network-metrics-daemon-km5cq\" (UID: \"b2780b90-9d95-44d8-bc10-8294255f08a5\") " pod="openshift-multus/network-metrics-daemon-km5cq" Oct 30 18:32:09.809845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.809827 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.821371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.821346 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbdj5\" (UniqueName: \"kubernetes.io/projected/dbee4f62-3043-46ce-821a-b77f3746f83a-kube-api-access-dbdj5\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.822084 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.822067 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"prometheus-k8s-0\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:09.823264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.823246 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"console-6ff767d568-st4n4\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:09.823582 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.823566 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2pws\" (UniqueName: \"kubernetes.io/projected/0c7fe3f2-f764-4843-8732-f5434a5f82cc-kube-api-access-g2pws\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.825198 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.825175 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt8jb\" (UniqueName: \"kubernetes.io/projected/6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b-kube-api-access-xt8jb\") pod \"thanos-querier-67ddcc4d7f-rndt7\" (UID: \"6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b\") " pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:09.827404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.827385 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9z8f\" (UniqueName: \"kubernetes.io/projected/1f3423a0-15ee-4a92-a819-0507cf101d2c-kube-api-access-m9z8f\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.827957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.827937 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh4g6\" (UniqueName: \"kubernetes.io/projected/4bd5c23b-225b-4eff-8b03-77fe7c04e56c-kube-api-access-fh4g6\") pod \"openshift-state-metrics-7bd7489957-w6tdb\" (UID: \"4bd5c23b-225b-4eff-8b03-77fe7c04e56c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-w6tdb" Oct 30 18:32:09.828269 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.828254 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnz9\" (UniqueName: \"kubernetes.io/projected/b3b05773-655d-4d2e-a924-2a6746e2a1e6-kube-api-access-nqnz9\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.834195 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.834166 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g95f\" (UniqueName: \"kubernetes.io/projected/502b8dd3-61d1-4a04-883e-35cf6460a661-kube-api-access-7g95f\") pod \"dns-default-c8dqv\" (UID: \"502b8dd3-61d1-4a04-883e-35cf6460a661\") " pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:09.835086 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.835060 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjb5\" (UniqueName: \"kubernetes.io/projected/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-kube-api-access-2vjb5\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.835402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.835383 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzc4f\" (UniqueName: \"kubernetes.io/projected/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-kube-api-access-qzc4f\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.836308 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.836279 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj4vw\" (UniqueName: \"kubernetes.io/projected/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-kube-api-access-qj4vw\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.837104 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.837083 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4mx\" (UniqueName: \"kubernetes.io/projected/852071ed-f1b5-4b6e-8ef5-1deb88d86855-kube-api-access-gf4mx\") pod \"metrics-server-7969db6d87-blhz9\" (UID: \"852071ed-f1b5-4b6e-8ef5-1deb88d86855\") " pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:09.837998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.837973 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66z2r\" (UniqueName: \"kubernetes.io/projected/24cc86a6-a954-4a74-a70b-63e31860233c-kube-api-access-66z2r\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.838079 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.838030 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"console-7cbcd65b88-x42qd\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:09.838620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.838603 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gmg\" (UniqueName: \"kubernetes.io/projected/195ffa97-91dc-4cad-802d-97027141794c-kube-api-access-v7gmg\") pod \"network-check-target-x7rq7\" (UID: \"195ffa97-91dc-4cad-802d-97027141794c\") " pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:09.839197 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.839173 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhhhq\" (UniqueName: \"kubernetes.io/projected/3dc7152c-07ad-466c-aed1-1e289c384fde-kube-api-access-zhhhq\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.839337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.839319 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcmw8\" (UniqueName: \"kubernetes.io/projected/a3b81f81-6c7f-44cd-a040-e7488cfea341-kube-api-access-lcmw8\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.891468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891368 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891404 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891429 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.891468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891462 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891490 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891496 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-slash\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891511 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891496 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-kubelet\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891523 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-cnibin\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891521 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-socket-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891574 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891589 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-etc-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891621 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891645 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-lib-modules\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891661 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-ovn\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891667 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891686 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891700 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891694 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-bin\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891731 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891750 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891756 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891769 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891778 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.891794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891792 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891808 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-var-lib-kubelet\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891821 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-hostroot\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891823 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891845 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891847 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-socket-dir-parent\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891859 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891871 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-multus-certs\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891883 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7a8c1d-004b-423c-8d4c-27e2b6757aae-host\") pod \"node-ca-fx675\" (UID: \"cf7a8c1d-004b-423c-8d4c-27e2b6757aae\") " pod="openshift-image-registry/node-ca-fx675" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891908 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891934 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891955 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891909 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-cni-netd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891978 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.891995 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-conf\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892018 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-systemd\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892026 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-host\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892018 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-kubelet\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892053 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892114 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-systemd\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892082 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892477 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.892523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892519 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892639 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892689 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-k8s-cni-cncf-io\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892729 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-run-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892586 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-run-netns\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892832 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892877 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892910 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892953 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.892980 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893026 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893060 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893104 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893138 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893161 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893205 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893235 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893276 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893306 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893344 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893373 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.893414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893416 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893445 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893466 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893505 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893566 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893598 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893638 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893667 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893698 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893737 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893768 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893807 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893831 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893869 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.893897 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894039 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-root\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894119 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894179 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3b05773-655d-4d2e-a924-2a6746e2a1e6-system-cni-dir\") pod \"multus-additional-cni-plugins-bpfcd\" (UID: \"b3b05773-655d-4d2e-a924-2a6746e2a1e6\") " pod="openshift-multus/multus-additional-cni-plugins-bpfcd" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894277 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-registration-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894323 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-systemd-units\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894388 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-kubernetes\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.894454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894456 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-device-dir\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894520 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-sys\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894601 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e025fb6d-2541-4243-a8d3-8fc4fc5f5940-hosts-file\") pod \"node-resolver-9r555\" (UID: \"e025fb6d-2541-4243-a8d3-8fc4fc5f5940\") " pod="openshift-dns/node-resolver-9r555" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894662 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-multus-conf-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894722 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-sys\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894802 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dc7152c-07ad-466c-aed1-1e289c384fde-node-exporter-wtmp\") pod \"node-exporter-24l9n\" (UID: \"3dc7152c-07ad-466c-aed1-1e289c384fde\") " pod="openshift-monitoring/node-exporter-24l9n" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894853 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-run\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894913 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-node-log\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.894968 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-ovn-kubernetes\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895040 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysconfig\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895105 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7fe3f2-f764-4843-8732-f5434a5f82cc-crio-socket\") pod \"insights-runtime-extractor-w9swt\" (UID: \"0c7fe3f2-f764-4843-8732-f5434a5f82cc\") " pod="openshift-insights/insights-runtime-extractor-w9swt" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895152 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-bin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895211 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-host-run-netns\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895269 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-etc-kubernetes\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.895341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895343 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895388 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3-host-slash\") pod \"iptables-alerter-5xlpr\" (UID: \"346d01a4-c662-4cb1-bdd7-a2e39ee5c6c3\") " pod="openshift-network-operator/iptables-alerter-5xlpr" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895511 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-modprobe-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895572 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-host-var-lib-cni-multus\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895636 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-kubelet-config\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895695 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f3423a0-15ee-4a92-a819-0507cf101d2c-sys-fs\") pod \"aws-ebs-csi-driver-node-ztn44\" (UID: \"1f3423a0-15ee-4a92-a819-0507cf101d2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztn44" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895790 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-log-socket\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895844 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-cnibin\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895899 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dbee4f62-3043-46ce-821a-b77f3746f83a-var-lib-openvswitch\") pod \"ovnkube-node-6v4xj\" (UID: \"dbee4f62-3043-46ce-821a-b77f3746f83a\") " pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:09.895998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.895972 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a56c3ed8-1972-402c-8195-b4dc842c69fb-dbus\") pod \"global-pull-secret-syncer-t6kkd\" (UID: \"a56c3ed8-1972-402c-8195-b4dc842c69fb\") " pod="kube-system/global-pull-secret-syncer-t6kkd" Oct 30 18:32:09.896353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.896038 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b81f81-6c7f-44cd-a040-e7488cfea341-system-cni-dir\") pod \"multus-vsk5n\" (UID: \"a3b81f81-6c7f-44cd-a040-e7488cfea341\") " pod="openshift-multus/multus-vsk5n" Oct 30 18:32:09.896659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:09.896633 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24cc86a6-a954-4a74-a70b-63e31860233c-etc-sysctl-d\") pod \"tuned-mlq92\" (UID: \"24cc86a6-a954-4a74-a70b-63e31860233c\") " pod="openshift-cluster-node-tuning-operator/tuned-mlq92" Oct 30 18:32:11.185923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:11.185891 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:12.760662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:12.760630 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c8dqv" Oct 30 18:32:13.036373 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:13.036297 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67ddcc4d7f-rndt7" Oct 30 18:32:13.617244 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:13.617202 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:13.632485 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:13.632462 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:13.834412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:13.834385 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:14.886201 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:14.886174 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:16.971829 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:16.971789 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:32:18.423882 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:18.423845 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:18.424267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:18.424090 6492 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:32:19.362324 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:19.362209 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:19.789063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:19.789038 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6v4xj" Oct 30 18:32:21.191498 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:21.191457 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:21.195258 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:21.195238 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:32:27.829794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:27.829757 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rl74h" Oct 30 18:32:33.922052 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:33.922017 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7969db6d87-blhz9" Oct 30 18:32:34.693631 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.693583 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6ff767d568-st4n4" podUID="6487588c-2589-404f-8711-3ccf6a999ebb" containerName="console" containerID="cri-o://45bcca9b5d9258f9fe4597278f6bd66f0cbbd9f82ea59b470b266186fd148a9f" gracePeriod=15 Oct 30 18:32:34.822110 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.822080 6492 generic.go:356] "Generic (PLEG): container finished" podID="6487588c-2589-404f-8711-3ccf6a999ebb" containerID="45bcca9b5d9258f9fe4597278f6bd66f0cbbd9f82ea59b470b266186fd148a9f" exitCode=0 Oct 30 18:32:34.822270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.822136 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff767d568-st4n4" event={"ID":"6487588c-2589-404f-8711-3ccf6a999ebb","Type":"ContainerDied","Data":"45bcca9b5d9258f9fe4597278f6bd66f0cbbd9f82ea59b470b266186fd148a9f"} Oct 30 18:32:34.968247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.968220 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:34.994737 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994706 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994750 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994781 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994800 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994819 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994849 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994877 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.994924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.994919 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") pod \"6487588c-2589-404f-8711-3ccf6a999ebb\" (UID: \"6487588c-2589-404f-8711-3ccf6a999ebb\") " Oct 30 18:32:34.995284 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.995073 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:34.995284 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.995134 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:34.995389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.995233 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:34.995389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.995356 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:34.995483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.995426 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:34.997296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.997273 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "kube-api-access-mhf4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:34.997385 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.997302 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:34.997425 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:34.997379 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config" pod "6487588c-2589-404f-8711-3ccf6a999ebb" (UID: "6487588c-2589-404f-8711-3ccf6a999ebb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:35.095807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095766 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhf4n\" (UniqueName: \"kubernetes.io/projected/6487588c-2589-404f-8711-3ccf6a999ebb-kube-api-access-mhf4n\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.095807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095803 6492 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-console-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.095807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095813 6492 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-oauth-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.096044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095821 6492 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-trusted-ca-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.096044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095832 6492 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6487588c-2589-404f-8711-3ccf6a999ebb-tmp\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.096044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095841 6492 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-oauth-serving-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.096044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095850 6492 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6487588c-2589-404f-8711-3ccf6a999ebb-console-serving-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.096044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.095859 6492 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6487588c-2589-404f-8711-3ccf6a999ebb-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.179080 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.179043 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-694f69bf4f-tpldd"] Oct 30 18:32:35.179401 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.179375 6492 cpu_manager.go:414] "containerMap: removing container" podUID="6487588c-2589-404f-8711-3ccf6a999ebb" containerName="console" Oct 30 18:32:35.179401 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.179393 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6487588c-2589-404f-8711-3ccf6a999ebb" containerName="console" Oct 30 18:32:35.179626 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.179456 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="6487588c-2589-404f-8711-3ccf6a999ebb" containerName="console" Oct 30 18:32:35.214111 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.214069 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694f69bf4f-tpldd"] Oct 30 18:32:35.214263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.214184 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297034 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.296996 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-service-ca\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297034 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297036 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9frhk\" (UniqueName: \"kubernetes.io/projected/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-kube-api-access-9frhk\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297246 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297128 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-tmp\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297246 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297161 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297246 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297212 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-trusted-ca-bundle\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297246 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297240 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297254 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-oauth-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.297372 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.297295 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-oauth-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398478 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-service-ca\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398516 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9frhk\" (UniqueName: \"kubernetes.io/projected/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-kube-api-access-9frhk\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398667 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-tmp\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398719 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398752 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-trusted-ca-bundle\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398785 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398869 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-oauth-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.398971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.398917 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-oauth-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.399096 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399070 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-tmp\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.399199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399181 6492 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:32:35.399298 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399278 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-service-ca\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.399552 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399521 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.399670 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399652 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-oauth-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.399787 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.399768 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-trusted-ca-bundle\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.401289 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.401270 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-serving-cert\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.401371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.401318 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-console-oauth-config\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.420823 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.420797 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9frhk\" (UniqueName: \"kubernetes.io/projected/4cfb133a-442c-48f2-a4c0-0cb30e64f7d8-kube-api-access-9frhk\") pod \"console-694f69bf4f-tpldd\" (UID: \"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8\") " pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.537322 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.537234 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:35.827354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.827282 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff767d568-st4n4" Oct 30 18:32:35.827354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.827289 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff767d568-st4n4" event={"ID":"6487588c-2589-404f-8711-3ccf6a999ebb","Type":"ContainerDied","Data":"3eceff6c5901e4778134a7c337b57e78d032751daacf4294ec44ee4207c6ac96"} Oct 30 18:32:35.827354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.827345 6492 scope.go:122] "RemoveContainer" containerID="45bcca9b5d9258f9fe4597278f6bd66f0cbbd9f82ea59b470b266186fd148a9f" Oct 30 18:32:35.834205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.834183 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694f69bf4f-tpldd"] Oct 30 18:32:35.839501 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:35.839474 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfb133a_442c_48f2_a4c0_0cb30e64f7d8.slice/crio-dc7c63d602c74b9776d7e78bae17a0166cbd278cda2336a20e2f9964712aea4c WatchSource:0}: Error finding container dc7c63d602c74b9776d7e78bae17a0166cbd278cda2336a20e2f9964712aea4c: Status 404 returned error can't find the container with id dc7c63d602c74b9776d7e78bae17a0166cbd278cda2336a20e2f9964712aea4c Oct 30 18:32:35.873220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.873053 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:32:35.878850 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:35.878825 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ff767d568-st4n4"] Oct 30 18:32:36.662999 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:36.662964 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6487588c-2589-404f-8711-3ccf6a999ebb" path="/var/lib/kubelet/pods/6487588c-2589-404f-8711-3ccf6a999ebb/volumes" Oct 30 18:32:36.832902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:36.832859 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f69bf4f-tpldd" event={"ID":"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8","Type":"ContainerStarted","Data":"1ffd4af0c11aaf0ad2531a8ab6a2be86402be4c0e7d8fdc140a73212d95ec6fb"} Oct 30 18:32:36.833067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:36.832908 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694f69bf4f-tpldd" event={"ID":"4cfb133a-442c-48f2-a4c0-0cb30e64f7d8","Type":"ContainerStarted","Data":"dc7c63d602c74b9776d7e78bae17a0166cbd278cda2336a20e2f9964712aea4c"} Oct 30 18:32:36.871886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:36.871835 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-694f69bf4f-tpldd" podStartSLOduration=1.871822568 podStartE2EDuration="1.871822568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:36.871420571 +0000 UTC m=+28.315811566" watchObservedRunningTime="2026-10-30 18:32:36.871822568 +0000 UTC m=+28.316213550" Oct 30 18:32:39.365509 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:39.365476 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x7rq7" Oct 30 18:32:39.782623 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:39.782586 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:44.336283 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336247 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336649 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="prometheus" containerID="cri-o://845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" gracePeriod=600 Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336668 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy" containerID="cri-o://2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" gracePeriod=600 Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336691 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" gracePeriod=600 Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336706 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-web" containerID="cri-o://26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" gracePeriod=600 Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336751 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="config-reloader" containerID="cri-o://63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" gracePeriod=600 Oct 30 18:32:44.336816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.336708 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="thanos-sidecar" containerID="cri-o://6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" gracePeriod=600 Oct 30 18:32:44.617457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.617416 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:44.681925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.681896 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.681937 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.681962 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682023 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682062 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682095 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682144 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682182 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682230 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682260 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682293 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682337 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682374 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682411 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682449 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682491 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682482 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682531 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682571 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682583 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682616 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.682933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682649 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") pod \"19921861-8bf5-4a85-b934-56a1b07cf4ae\" (UID: \"19921861-8bf5-4a85-b934-56a1b07cf4ae\") " Oct 30 18:32:44.683331 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682960 6492 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.683331 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.682987 6492 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.683437 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.683408 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.683492 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.683445 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.683851 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.683830 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.686312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.684426 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.686312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.686052 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:44.687000 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.686896 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.687122 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.687067 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.687341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.687317 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.687433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.687329 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:44.688655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.688518 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.688655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.688513 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:44.688821 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.688733 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "kube-api-access-6kf9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:44.688996 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.688973 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:44.689091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.688971 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.689091 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.689060 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.689182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.689120 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.689290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.689269 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.697390 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.697367 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config" pod "19921861-8bf5-4a85-b934-56a1b07cf4ae" (UID: "19921861-8bf5-4a85-b934-56a1b07cf4ae"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:44.783426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783391 6492 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783420 6492 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783432 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kf9p\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-kube-api-access-6kf9p\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783443 6492 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-db\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783452 6492 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-metrics-client-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783461 6492 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783470 6492 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19921861-8bf5-4a85-b934-56a1b07cf4ae-config-out\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783479 6492 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783488 6492 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783497 6492 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-configmap-metrics-client-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783507 6492 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-kube-rbac-proxy\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783515 6492 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19921861-8bf5-4a85-b934-56a1b07cf4ae-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783523 6492 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783532 6492 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19921861-8bf5-4a85-b934-56a1b07cf4ae-tls-assets\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783560 6492 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-web-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783568 6492 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783577 6492 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-secret-grpc-tls\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.783671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.783585 6492 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/19921861-8bf5-4a85-b934-56a1b07cf4ae-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865573 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865600 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865606 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865612 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865619 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865625 6492 generic.go:356] "Generic (PLEG): container finished" podID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" exitCode=0 Oct 30 18:32:44.865655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865652 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865653 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865739 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865782 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865801 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865823 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865864 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865908 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} Oct 30 18:32:44.866066 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.865950 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19921861-8bf5-4a85-b934-56a1b07cf4ae","Type":"ContainerDied","Data":"a93ec0ec9c8a6278d3e29b23d2bf572edb012ab1b4ff786f451f902302b7e1c2"} Oct 30 18:32:44.873852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.873687 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.881587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.881565 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.890413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.890393 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.897336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.897319 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.903740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.903721 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.910533 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.910509 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.914252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.914228 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:44.917920 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.917903 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.918199 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.918180 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.918250 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918209 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.918296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918251 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.918506 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.918491 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.918562 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918509 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.918562 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918523 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.918757 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.918738 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.918798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918763 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.918798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918779 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.918995 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.918979 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.919029 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.918999 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.919029 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919011 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.919228 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.919213 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.919281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919230 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.919281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919242 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.919464 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.919447 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.919502 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919466 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.919502 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919478 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.919955 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:44.919937 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.920017 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919957 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.920017 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.919968 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.920187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920171 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.920230 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920186 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.920392 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920376 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.920433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920391 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.920606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920589 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.920645 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920606 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.920795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920782 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.920837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920796 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.920985 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920968 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.921022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.920985 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.921163 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921149 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.921200 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921163 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.921373 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921357 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.921373 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921372 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.921766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921746 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.921766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921764 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.921973 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921956 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.922025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.921973 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.922233 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922214 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.922305 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922234 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.922495 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922477 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.922556 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922495 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.922741 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922721 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.922741 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922740 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.922942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922927 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.922942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.922941 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.923177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923161 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.923217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923176 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.923267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923213 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:44.923412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923393 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.923458 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923420 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.923700 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923684 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.923748 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923700 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.923892 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923877 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.923932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.923892 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.924077 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924061 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.924120 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924077 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.924281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924260 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.924334 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924282 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.924482 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924468 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.924559 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924482 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.924724 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924709 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.924724 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924723 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.924949 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924931 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.925002 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.924949 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.925163 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925143 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.925203 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925164 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.925382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925366 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.925382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925382 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.925594 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925573 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.925657 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925594 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.925809 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925791 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.925862 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.925809 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.926020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926004 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.926020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926019 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.926232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926216 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.926297 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926231 6492 scope.go:122] "RemoveContainer" containerID="5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6" Oct 30 18:32:44.926483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926466 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6"} err="failed to get container status \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": rpc error: code = NotFound desc = could not find container \"5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6\": container with ID starting with 5c3bed8b201e6e39c44c423f8b61387d7c24de673b64eedaa893a02861c4bee6 not found: ID does not exist" Oct 30 18:32:44.926548 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926483 6492 scope.go:122] "RemoveContainer" containerID="2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62" Oct 30 18:32:44.926812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926725 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62"} err="failed to get container status \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": rpc error: code = NotFound desc = could not find container \"2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62\": container with ID starting with 2711ad541940aa41a7b558767f69ee9b4e5fb230fd909415ec0ff6ccd68b9a62 not found: ID does not exist" Oct 30 18:32:44.926812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.926747 6492 scope.go:122] "RemoveContainer" containerID="26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7" Oct 30 18:32:44.927125 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927108 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7"} err="failed to get container status \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": rpc error: code = NotFound desc = could not find container \"26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7\": container with ID starting with 26eedf090b1629043bb54623d2de86037692ac604babe9c8ab04f1cb1a2a14c7 not found: ID does not exist" Oct 30 18:32:44.927125 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927125 6492 scope.go:122] "RemoveContainer" containerID="6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24" Oct 30 18:32:44.927343 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927328 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24"} err="failed to get container status \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": rpc error: code = NotFound desc = could not find container \"6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24\": container with ID starting with 6763504f79ef6543e40c75c6db3550d3624f70d407134f06ed11832be00abe24 not found: ID does not exist" Oct 30 18:32:44.927343 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927342 6492 scope.go:122] "RemoveContainer" containerID="63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59" Oct 30 18:32:44.927533 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927516 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59"} err="failed to get container status \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": rpc error: code = NotFound desc = could not find container \"63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59\": container with ID starting with 63ba5b4dd7870887ae59ad06600326a5375e67c7c59ec5bf3fb4ae2b1ab6cb59 not found: ID does not exist" Oct 30 18:32:44.927533 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927532 6492 scope.go:122] "RemoveContainer" containerID="845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac" Oct 30 18:32:44.927805 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927779 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac"} err="failed to get container status \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": rpc error: code = NotFound desc = could not find container \"845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac\": container with ID starting with 845d519269a5583b080fcf215df1aa17827827d3317ffe6365cbd2783c5639ac not found: ID does not exist" Oct 30 18:32:44.927905 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.927806 6492 scope.go:122] "RemoveContainer" containerID="23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823" Oct 30 18:32:44.928106 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.928086 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823"} err="failed to get container status \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": rpc error: code = NotFound desc = could not find container \"23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823\": container with ID starting with 23138d80f7a93297f12bd2fc6ae5bd6a416b4e80258ab669a2079e8414c3c823 not found: ID does not exist" Oct 30 18:32:44.954915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.954883 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:44.955188 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955173 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-web" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955201 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-web" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955215 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="prometheus" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955221 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="prometheus" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955227 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="init-config-reloader" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955233 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="init-config-reloader" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955252 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-thanos" Oct 30 18:32:44.955257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955258 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-thanos" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955266 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="config-reloader" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955270 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="config-reloader" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955282 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955286 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955298 6492 cpu_manager.go:414] "containerMap: removing container" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="thanos-sidecar" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955302 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="thanos-sidecar" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955344 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="config-reloader" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955370 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955377 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="thanos-sidecar" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955382 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-web" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955388 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="prometheus" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955393 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="init-config-reloader" Oct 30 18:32:44.955511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.955400 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" containerName="kube-rbac-proxy-thanos" Oct 30 18:32:44.959665 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.959650 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:44.973221 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.973201 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 18:32:44.973318 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.973229 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 18:32:44.973637 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.973623 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 18:32:44.984522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984499 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 18:32:44.984631 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984606 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 18:32:44.984704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984633 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 18:32:44.984704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984653 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 18:32:44.984704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984637 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 18:32:44.984836 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984718 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 18:32:44.984836 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984742 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8lqh3egd5j83h\"" Oct 30 18:32:44.984836 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984720 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 18:32:44.984836 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984782 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 18:32:44.984836 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984805 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bqjtk\"" Oct 30 18:32:44.985063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984871 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 18:32:44.985063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.984968 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 18:32:44.985063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.985034 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 18:32:44.990772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:44.990747 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:45.084846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.084807 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.084846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.084850 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.084876 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.084967 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085006 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085023 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085053 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085082 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q5kp\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-kube-api-access-9q5kp\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085160 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085178 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085197 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085318 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085276 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085318 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085296 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085378 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085318 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085378 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085332 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085378 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085356 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085393 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-web-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085448 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-config-out\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085472 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.085505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.085489 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186489 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186395 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186489 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186452 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186585 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186632 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186651 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q5kp\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-kube-api-access-9q5kp\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186683 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186887 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186709 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186887 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186810 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.186887 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186874 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186910 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186944 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186969 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.186993 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.187018 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-web-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187040 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.187021 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187338 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.187284 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.187338 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.187284 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.188038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.187867 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189619 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189590 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-web-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189638 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189643 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-config-out\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189677 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189709 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189737 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189744 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189755 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189786 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.189933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.189808 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.190185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.190079 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.190812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.190487 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.190812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.190613 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.190812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.190785 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.191119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.191094 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.191249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.191220 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79454a9c-ca82-4d0e-ba17-e2375e945f46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.191417 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.191229 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.191992 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.191968 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79454a9c-ca82-4d0e-ba17-e2375e945f46-config-out\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.192076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.192028 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.192207 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.192187 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-config\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.192661 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.192642 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79454a9c-ca82-4d0e-ba17-e2375e945f46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.208496 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.208478 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q5kp\" (UniqueName: \"kubernetes.io/projected/79454a9c-ca82-4d0e-ba17-e2375e945f46-kube-api-access-9q5kp\") pod \"prometheus-k8s-0\" (UID: \"79454a9c-ca82-4d0e-ba17-e2375e945f46\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.281618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.281574 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:32:45.538345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.538291 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:45.538776 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.538370 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:45.543360 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.543336 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:45.618266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.618230 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 18:32:45.622918 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:32:45.622890 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79454a9c_ca82_4d0e_ba17_e2375e945f46.slice/crio-1f16be4aa808f331058bc93996324b999085e55e8e3e4c42f1d1c6471f3c2ebe WatchSource:0}: Error finding container 1f16be4aa808f331058bc93996324b999085e55e8e3e4c42f1d1c6471f3c2ebe: Status 404 returned error can't find the container with id 1f16be4aa808f331058bc93996324b999085e55e8e3e4c42f1d1c6471f3c2ebe Oct 30 18:32:45.871687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.871601 6492 generic.go:356] "Generic (PLEG): container finished" podID="79454a9c-ca82-4d0e-ba17-e2375e945f46" containerID="ce50c3e4e850fa6e0811553c282fefe55300bef5ca4d8c7fabc1a92031c456dc" exitCode=0 Oct 30 18:32:45.871841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.871688 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerDied","Data":"ce50c3e4e850fa6e0811553c282fefe55300bef5ca4d8c7fabc1a92031c456dc"} Oct 30 18:32:45.871841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.871781 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"1f16be4aa808f331058bc93996324b999085e55e8e3e4c42f1d1c6471f3c2ebe"} Oct 30 18:32:45.876681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.876655 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-694f69bf4f-tpldd" Oct 30 18:32:45.995304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:45.995277 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:32:46.663606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.663569 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19921861-8bf5-4a85-b934-56a1b07cf4ae" path="/var/lib/kubelet/pods/19921861-8bf5-4a85-b934-56a1b07cf4ae/volumes" Oct 30 18:32:46.880868 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.880826 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"d68de3f76dc1ac4c7dc7b910362faf75eeca3c73f51c462d4cc1ab41522389b1"} Oct 30 18:32:46.881015 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.880898 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"44b721a7d20dce697322a299cca5bd2470544b8287e62a7399cc1118494139fd"} Oct 30 18:32:46.881015 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.880930 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"7dff1e1e1235180b72389477e1f2d238eaf41937dcba6e668d3b05ec9438efdb"} Oct 30 18:32:46.881015 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.880994 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"a60172a872dc1f26570d507c663816c334c31d8f687103c70e04300db57dea07"} Oct 30 18:32:46.881176 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.881019 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"fbee07f95594ca6c75f3ef83696d934ec39db82817e2c03681a3c80a2cdfd78c"} Oct 30 18:32:46.881176 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.881043 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79454a9c-ca82-4d0e-ba17-e2375e945f46","Type":"ContainerStarted","Data":"dd6ddec4429119a3aa68bd2fa3ebd991f0f9154b88b578ba079de31170aa1963"} Oct 30 18:32:46.945193 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:46.945078 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.926472946 podStartE2EDuration="2.945064061s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.591115ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:46.944899492 +0000 UTC m=+38.389290489" watchObservedRunningTime="2026-10-30 18:32:46.945064061 +0000 UTC m=+38.389455053" Oct 30 18:32:46.972653 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:32:46.972616 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:32:50.282309 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:32:50.282268 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:33:02.938181 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:02.938147 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:33:02.953408 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:02.953377 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:33:03.954938 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:03.954911 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 18:33:11.019805 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.019742 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7cbcd65b88-x42qd" podUID="8a9150d1-1c80-4155-9b40-64793e254389" containerName="console" containerID="cri-o://45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771" gracePeriod=15 Oct 30 18:33:11.271266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.271214 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:33:11.423874 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.423832 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.423874 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.423880 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424139 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.423903 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424139 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.423935 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424139 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.423953 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424139 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424043 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424152 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424194 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") pod \"8a9150d1-1c80-4155-9b40-64793e254389\" (UID: \"8a9150d1-1c80-4155-9b40-64793e254389\") " Oct 30 18:33:11.424355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424198 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:33:11.424501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424346 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:11.424627 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424500 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:11.424691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424614 6492 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-trusted-ca-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.424691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424648 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:11.424691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424664 6492 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a9150d1-1c80-4155-9b40-64793e254389-tmp\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.424810 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.424711 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:11.426346 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.426315 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:33:11.426464 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.426398 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:33:11.426464 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.426402 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc" pod "8a9150d1-1c80-4155-9b40-64793e254389" (UID: "8a9150d1-1c80-4155-9b40-64793e254389"). InnerVolumeSpecName "kube-api-access-d2rcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.525999 6492 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.526035 6492 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-serving-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.526045 6492 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-console-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.526054 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2rcc\" (UniqueName: \"kubernetes.io/projected/8a9150d1-1c80-4155-9b40-64793e254389-kube-api-access-d2rcc\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.526064 6492 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9150d1-1c80-4155-9b40-64793e254389-oauth-serving-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.526092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.526073 6492 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9150d1-1c80-4155-9b40-64793e254389-console-oauth-config\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:33:11.968481 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.968449 6492 generic.go:356] "Generic (PLEG): container finished" podID="8a9150d1-1c80-4155-9b40-64793e254389" containerID="45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771" exitCode=0 Oct 30 18:33:11.968656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.968497 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcd65b88-x42qd" Oct 30 18:33:11.968656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.968530 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcd65b88-x42qd" event={"ID":"8a9150d1-1c80-4155-9b40-64793e254389","Type":"ContainerDied","Data":"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771"} Oct 30 18:33:11.968656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.968587 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcd65b88-x42qd" event={"ID":"8a9150d1-1c80-4155-9b40-64793e254389","Type":"ContainerDied","Data":"000f16ddec881ac867570f4aa53231a8a3e5280c2dedc5ca98e742f26cd66ea2"} Oct 30 18:33:11.968656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.968607 6492 scope.go:122] "RemoveContainer" containerID="45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771" Oct 30 18:33:11.977281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.977263 6492 scope.go:122] "RemoveContainer" containerID="45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771" Oct 30 18:33:11.977567 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:33:11.977518 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771\": container with ID starting with 45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771 not found: ID does not exist" containerID="45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771" Oct 30 18:33:11.977633 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:11.977568 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771"} err="failed to get container status \"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771\": rpc error: code = NotFound desc = could not find container \"45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771\": container with ID starting with 45b458a9eb549d89976610b722ae40288d9d4002365a67857f8ec54ba050a771 not found: ID does not exist" Oct 30 18:33:12.012857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:12.012829 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:33:12.017099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:12.017077 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cbcd65b88-x42qd"] Oct 30 18:33:12.193650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:12.193607 6492 patch_prober.go:29] interesting pod/console-7cbcd65b88-x42qd container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.14:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:33:12.194035 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:12.193680 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/console-7cbcd65b88-x42qd" podUID="8a9150d1-1c80-4155-9b40-64793e254389" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 18:33:12.662246 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:33:12.662211 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a9150d1-1c80-4155-9b40-64793e254389" path="/var/lib/kubelet/pods/8a9150d1-1c80-4155-9b40-64793e254389/volumes" Oct 30 18:33:16.971320 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:33:16.971284 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:33:46.972088 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:33:46.972052 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:34:16.971632 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:34:16.971595 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:34:46.971203 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:34:46.971166 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:35:16.970728 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:35:16.970687 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:35:17.453345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.453316 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m"] Oct 30 18:35:17.455922 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.455855 6492 cpu_manager.go:414] "containerMap: removing container" podUID="8a9150d1-1c80-4155-9b40-64793e254389" containerName="console" Oct 30 18:35:17.455922 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.455887 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a9150d1-1c80-4155-9b40-64793e254389" containerName="console" Oct 30 18:35:17.456439 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.456320 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a9150d1-1c80-4155-9b40-64793e254389" containerName="console" Oct 30 18:35:17.460432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.460412 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.489154 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.489123 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Oct 30 18:35:17.489295 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.489284 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Oct 30 18:35:17.489362 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.489336 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-2ckgm\"" Oct 30 18:35:17.490363 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.490337 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m"] Oct 30 18:35:17.615389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.615346 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.615389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.615386 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.615656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.615492 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfptq\" (UniqueName: \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.716426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.716350 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.716426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.716388 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.716624 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.716434 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfptq\" (UniqueName: \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.716783 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.716753 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.716825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.716775 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.738664 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.738642 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfptq\" (UniqueName: \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:17.774996 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:17.774971 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:18.070436 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:18.070342 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m"] Oct 30 18:35:18.075489 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:18.075459 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454162e5_cc43_4798_8490_2efcc954f558.slice/crio-b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921 WatchSource:0}: Error finding container b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921: Status 404 returned error can't find the container with id b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921 Oct 30 18:35:18.077314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:18.077293 6492 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:35:18.380943 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:18.380866 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerStarted","Data":"b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921"} Oct 30 18:35:20.054589 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.054488 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss"] Oct 30 18:35:20.058071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.058048 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.081661 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.081616 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-qzkf9\"" Oct 30 18:35:20.082660 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.082630 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss"] Oct 30 18:35:20.134996 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.134959 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.135173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.135094 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.135223 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.135193 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5495p\" (UniqueName: \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.155726 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.155693 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc"] Oct 30 18:35:20.159522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.159497 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.172785 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.172762 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-nngbh\"" Oct 30 18:35:20.183244 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.183221 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc"] Oct 30 18:35:20.235685 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235652 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5495p\" (UniqueName: \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.235685 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235690 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.235907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235744 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.235907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235776 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.235907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235802 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qf4c\" (UniqueName: \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.236032 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.235926 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.236220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.236199 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.236263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.236216 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.258039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.258009 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6"] Oct 30 18:35:20.258242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.258220 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5495p\" (UniqueName: \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.261642 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.261620 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.274487 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.274460 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-q6dzl\"" Oct 30 18:35:20.283392 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.283366 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6"] Oct 30 18:35:20.336596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336502 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.336596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336570 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czr2\" (UniqueName: \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.336817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336613 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.336817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336671 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qf4c\" (UniqueName: \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.336817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336722 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.336817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.336748 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.337159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.337139 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.337222 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.337167 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.355760 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.355734 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth"] Oct 30 18:35:20.359302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.359281 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.363990 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.363971 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qf4c\" (UniqueName: \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.375054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.375016 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:20.375428 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.375407 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-pfrlr\"" Oct 30 18:35:20.391690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.391666 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth"] Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437436 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437476 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czr2\" (UniqueName: \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437497 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437520 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbv78\" (UniqueName: \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437637 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.437684 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.438120 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.438169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.438144 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.460343 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.460282 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czr2\" (UniqueName: \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.477521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.477489 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:20.538938 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.538912 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.539054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.538969 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.539054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.539014 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbv78\" (UniqueName: \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.539348 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.539326 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.539593 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.539572 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.562137 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.562104 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbv78\" (UniqueName: \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.581782 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.581747 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:20.683886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.683851 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss"] Oct 30 18:35:20.685011 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.684986 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:20.688462 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:20.688434 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4138f84_dd0a_498c_b080_9e0b74466fad.slice/crio-539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc WatchSource:0}: Error finding container 539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc: Status 404 returned error can't find the container with id 539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc Oct 30 18:35:20.784660 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.784626 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc"] Oct 30 18:35:20.790089 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:20.790057 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2a1ef1_0062_4369_aacb_adf384934cc9.slice/crio-0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c WatchSource:0}: Error finding container 0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c: Status 404 returned error can't find the container with id 0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c Oct 30 18:35:20.890757 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.890669 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6"] Oct 30 18:35:20.895340 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:20.895309 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07b8e21_48ed_49ee_8f10_135063b21802.slice/crio-16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78 WatchSource:0}: Error finding container 16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78: Status 404 returned error can't find the container with id 16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78 Oct 30 18:35:20.991615 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:20.991576 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth"] Oct 30 18:35:20.997533 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:20.997503 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf059b75e_26e1_468c_9150_b6250de5e0ef.slice/crio-5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85 WatchSource:0}: Error finding container 5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85: Status 404 returned error can't find the container with id 5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85 Oct 30 18:35:21.236969 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.235051 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g"] Oct 30 18:35:21.239140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.239112 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.246014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.245767 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.246014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.245868 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.246014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.245894 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w242d\" (UniqueName: \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.253579 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.253530 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-88w5n\"" Oct 30 18:35:21.259437 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.259390 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g"] Oct 30 18:35:21.347812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.346697 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.347812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.346786 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.347812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.346818 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w242d\" (UniqueName: \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.347812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.347695 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.348233 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.348021 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.382769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.382707 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w242d\" (UniqueName: \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.413814 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.413770 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" event={"ID":"df2a1ef1-0062-4369-aacb-adf384934cc9","Type":"ContainerStarted","Data":"0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c"} Oct 30 18:35:21.418793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.418758 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" event={"ID":"f059b75e-26e1-468c-9150-b6250de5e0ef","Type":"ContainerStarted","Data":"5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85"} Oct 30 18:35:21.423986 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.423946 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" event={"ID":"d4138f84-dd0a-498c-b080-9e0b74466fad","Type":"ContainerStarted","Data":"539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc"} Oct 30 18:35:21.425852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.425824 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" event={"ID":"c07b8e21-48ed-49ee-8f10-135063b21802","Type":"ContainerStarted","Data":"16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78"} Oct 30 18:35:21.562805 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.562725 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:21.592720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.592692 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl"] Oct 30 18:35:21.598894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.598869 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.615457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.615414 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-m5nwz\"" Oct 30 18:35:21.617163 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.617093 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl"] Oct 30 18:35:21.649402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.649217 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.649402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.649267 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.649402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.649289 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2f9g\" (UniqueName: \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.751020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.749838 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.751020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.749904 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2f9g\" (UniqueName: \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.751020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.749975 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.751020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.750430 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.751020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.750678 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.773655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.773600 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2f9g\" (UniqueName: \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:21.900075 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.899988 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g"] Oct 30 18:35:21.908374 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:21.908343 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d212d8_ddc3_4498_af75_ac44b6553429.slice/crio-a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087 WatchSource:0}: Error finding container a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087: Status 404 returned error can't find the container with id a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087 Oct 30 18:35:21.918068 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:21.918046 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:22.101822 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.100364 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f"] Oct 30 18:35:22.105290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.105264 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.120590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.120563 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:35:22.120808 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.120697 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 18:35:22.120808 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.120736 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:35:22.120808 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.120800 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9bw4x\"" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.152831 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.152882 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.152918 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.152956 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.152987 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.153021 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwpdt\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-kube-api-access-nwpdt\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.153048 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.153078 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.153112 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.153456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.153225 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f"] Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254685 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254739 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254783 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpdt\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-kube-api-access-nwpdt\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254809 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254842 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254880 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.254964 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.255002 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.255698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.255036 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.257554 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.257331 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.257554 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.257503 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.264137 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.263887 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.264137 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.263949 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.264137 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.264099 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.264360 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.264184 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff54aab4-243f-4c0f-b941-5e2c6839ab77-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.264514 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.264457 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.268886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.268821 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl"] Oct 30 18:35:22.286217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.286190 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.288511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.288483 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpdt\" (UniqueName: \"kubernetes.io/projected/ff54aab4-243f-4c0f-b941-5e2c6839ab77-kube-api-access-nwpdt\") pod \"maas-default-gateway-openshift-default-6dcf757899-2bv6f\" (UID: \"ff54aab4-243f-4c0f-b941-5e2c6839ab77\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.424323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.423988 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:22.433845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.433761 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" event={"ID":"acafbd00-3e35-4007-9472-261c7f8b97a9","Type":"ContainerStarted","Data":"fdaaed66be79b0a7182e558d7086e5580413596726c9ed401666a871896ae828"} Oct 30 18:35:22.435407 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.435331 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerStarted","Data":"a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087"} Oct 30 18:35:22.793083 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:22.788020 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f"] Oct 30 18:35:22.796812 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:22.796763 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff54aab4_243f_4c0f_b941_5e2c6839ab77.slice/crio-3950757dae69c30019d49237b6549246dc91f03232a4cbae0bbb88c013364755 WatchSource:0}: Error finding container 3950757dae69c30019d49237b6549246dc91f03232a4cbae0bbb88c013364755: Status 404 returned error can't find the container with id 3950757dae69c30019d49237b6549246dc91f03232a4cbae0bbb88c013364755 Oct 30 18:35:23.448239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:23.448161 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" event={"ID":"ff54aab4-243f-4c0f-b941-5e2c6839ab77","Type":"ContainerStarted","Data":"3950757dae69c30019d49237b6549246dc91f03232a4cbae0bbb88c013364755"} Oct 30 18:35:29.509586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:29.508361 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerStarted","Data":"69193625da2c484783f28231812d00114b3f4a611ce26237c4443a3ddbeffc3b"} Oct 30 18:35:29.843724 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:29.843687 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:29.843831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:29.843781 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:29.843904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:29.843826 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:30.514472 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.514414 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" event={"ID":"ff54aab4-243f-4c0f-b941-5e2c6839ab77","Type":"ContainerStarted","Data":"ec1f0cf1ec8d7025f7e986d675c9d5e938df02b02e0b6a58eb9c5690f0e6298b"} Oct 30 18:35:30.517011 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.516983 6492 generic.go:356] "Generic (PLEG): container finished" podID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerID="5d31456a77620f06e5dd19b9d92460061a4a245837937e5b5fba1c906ed53e6e" exitCode=0 Oct 30 18:35:30.517160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.517066 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" event={"ID":"d4138f84-dd0a-498c-b080-9e0b74466fad","Type":"ContainerDied","Data":"5d31456a77620f06e5dd19b9d92460061a4a245837937e5b5fba1c906ed53e6e"} Oct 30 18:35:30.519837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.519810 6492 generic.go:356] "Generic (PLEG): container finished" podID="c07b8e21-48ed-49ee-8f10-135063b21802" containerID="3fe48ab8153b14c140ea17b481a9b8dc60d065fe3a5c17601ecd6e9b2abaa8c3" exitCode=0 Oct 30 18:35:30.519948 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.519902 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" event={"ID":"c07b8e21-48ed-49ee-8f10-135063b21802","Type":"ContainerDied","Data":"3fe48ab8153b14c140ea17b481a9b8dc60d065fe3a5c17601ecd6e9b2abaa8c3"} Oct 30 18:35:30.523417 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.523394 6492 generic.go:356] "Generic (PLEG): container finished" podID="454162e5-cc43-4798-8490-2efcc954f558" containerID="69193625da2c484783f28231812d00114b3f4a611ce26237c4443a3ddbeffc3b" exitCode=0 Oct 30 18:35:30.523742 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.523709 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerDied","Data":"69193625da2c484783f28231812d00114b3f4a611ce26237c4443a3ddbeffc3b"} Oct 30 18:35:30.526274 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.525988 6492 generic.go:356] "Generic (PLEG): container finished" podID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerID="8f163ed19ba11c006e10607624b5f461a48737f3d9fc8862699fab3a62ce8d68" exitCode=0 Oct 30 18:35:30.526274 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.526038 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" event={"ID":"f059b75e-26e1-468c-9150-b6250de5e0ef","Type":"ContainerDied","Data":"8f163ed19ba11c006e10607624b5f461a48737f3d9fc8862699fab3a62ce8d68"} Oct 30 18:35:30.568330 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:30.567313 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podStartSLOduration=1.526236467 podStartE2EDuration="8.567298729s" totalImagesPullingTime="7.041062262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:30.566916204 +0000 UTC m=+202.011307223" watchObservedRunningTime="2026-10-30 18:35:30.567298729 +0000 UTC m=+202.011689722" Oct 30 18:35:31.424520 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.424432 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:31.429934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.429909 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:31.534059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.534021 6492 generic.go:356] "Generic (PLEG): container finished" podID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerID="c0d20581c18a83bc4266bdf82cacccddb19706179e1b863c080a8bceaf9bef80" exitCode=0 Oct 30 18:35:31.534521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.534188 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" event={"ID":"f059b75e-26e1-468c-9150-b6250de5e0ef","Type":"ContainerDied","Data":"c0d20581c18a83bc4266bdf82cacccddb19706179e1b863c080a8bceaf9bef80"} Oct 30 18:35:31.536334 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.536305 6492 generic.go:356] "Generic (PLEG): container finished" podID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerID="c45e0bc8a8bdea0774f1d661b88c786c20c94442fc5f9063afbaff5c468ee5b7" exitCode=0 Oct 30 18:35:31.536432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.536380 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" event={"ID":"acafbd00-3e35-4007-9472-261c7f8b97a9","Type":"ContainerDied","Data":"c45e0bc8a8bdea0774f1d661b88c786c20c94442fc5f9063afbaff5c468ee5b7"} Oct 30 18:35:31.539776 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.539746 6492 generic.go:356] "Generic (PLEG): container finished" podID="06d212d8-ddc3-4498-af75-ac44b6553429" containerID="af95bd3bb5e194f3f2ee79ce750a41543d1ec6b0cffbd24577ffc22b3d258837" exitCode=0 Oct 30 18:35:31.540177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.539904 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerDied","Data":"af95bd3bb5e194f3f2ee79ce750a41543d1ec6b0cffbd24577ffc22b3d258837"} Oct 30 18:35:31.540177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.540050 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:31.541165 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:31.541145 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:35:32.549835 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.549703 6492 generic.go:356] "Generic (PLEG): container finished" podID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerID="9e49618c70fb0fc7b3819244c20551370c93507ec4828cecab27f80919f1426e" exitCode=0 Oct 30 18:35:32.549835 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.549797 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" event={"ID":"d4138f84-dd0a-498c-b080-9e0b74466fad","Type":"ContainerDied","Data":"9e49618c70fb0fc7b3819244c20551370c93507ec4828cecab27f80919f1426e"} Oct 30 18:35:32.552984 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.552959 6492 generic.go:356] "Generic (PLEG): container finished" podID="c07b8e21-48ed-49ee-8f10-135063b21802" containerID="f1f1eee6335ae55601abd76c8b86c6dad7e32a564ce83f32c4c15cf1e3240d7d" exitCode=0 Oct 30 18:35:32.553075 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.553033 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" event={"ID":"c07b8e21-48ed-49ee-8f10-135063b21802","Type":"ContainerDied","Data":"f1f1eee6335ae55601abd76c8b86c6dad7e32a564ce83f32c4c15cf1e3240d7d"} Oct 30 18:35:32.557610 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.557584 6492 generic.go:356] "Generic (PLEG): container finished" podID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerID="d411b6ea534089a4e75d67bd2e5e77e5383c9dee6079d8592408dd1802e4680a" exitCode=0 Oct 30 18:35:32.558199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:32.558149 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" event={"ID":"acafbd00-3e35-4007-9472-261c7f8b97a9","Type":"ContainerDied","Data":"d411b6ea534089a4e75d67bd2e5e77e5383c9dee6079d8592408dd1802e4680a"} Oct 30 18:35:33.574735 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:33.574681 6492 generic.go:356] "Generic (PLEG): container finished" podID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerID="b11cd25457562df550a61fd5387b8684f9c3dd498c5dbe502fb87eb588cfdf8c" exitCode=0 Oct 30 18:35:33.575240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:33.574739 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" event={"ID":"df2a1ef1-0062-4369-aacb-adf384934cc9","Type":"ContainerDied","Data":"b11cd25457562df550a61fd5387b8684f9c3dd498c5dbe502fb87eb588cfdf8c"} Oct 30 18:35:35.593691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:35.593625 6492 generic.go:356] "Generic (PLEG): container finished" podID="454162e5-cc43-4798-8490-2efcc954f558" containerID="88fa0b4ba9b4426872a70489cf265ac8b4ea6c2732988688c5f95e6ccc102f7c" exitCode=0 Oct 30 18:35:35.594722 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:35.594693 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerDied","Data":"88fa0b4ba9b4426872a70489cf265ac8b4ea6c2732988688c5f95e6ccc102f7c"} Oct 30 18:35:35.602140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:35.602106 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerStarted","Data":"9062dc58e34350e43436ac8b282e4fd6907c7ea46d2efb2062c1269d519d3810"} Oct 30 18:35:36.611409 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:36.611372 6492 generic.go:356] "Generic (PLEG): container finished" podID="06d212d8-ddc3-4498-af75-ac44b6553429" containerID="9062dc58e34350e43436ac8b282e4fd6907c7ea46d2efb2062c1269d519d3810" exitCode=0 Oct 30 18:35:36.611925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:36.611485 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerDied","Data":"9062dc58e34350e43436ac8b282e4fd6907c7ea46d2efb2062c1269d519d3810"} Oct 30 18:35:36.615494 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:36.615464 6492 generic.go:356] "Generic (PLEG): container finished" podID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerID="cdfa1cb887132fdf7df05bd1a38eb9e3b8845e48327b1eee74f157d32a6833af" exitCode=0 Oct 30 18:35:36.616276 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:36.616249 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" event={"ID":"df2a1ef1-0062-4369-aacb-adf384934cc9","Type":"ContainerDied","Data":"cdfa1cb887132fdf7df05bd1a38eb9e3b8845e48327b1eee74f157d32a6833af"} Oct 30 18:35:45.687041 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:45.687001 6492 generic.go:356] "Generic (PLEG): container finished" podID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerID="931db5249fd4ea547c323a9edc6e85cc7b657ac3c61fd8769ca11b56c00d7c63" exitCode=0 Oct 30 18:35:45.687570 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:45.687086 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" event={"ID":"f059b75e-26e1-468c-9150-b6250de5e0ef","Type":"ContainerDied","Data":"931db5249fd4ea547c323a9edc6e85cc7b657ac3c61fd8769ca11b56c00d7c63"} Oct 30 18:35:46.694131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.694094 6492 generic.go:356] "Generic (PLEG): container finished" podID="c07b8e21-48ed-49ee-8f10-135063b21802" containerID="7ac70a1df0f20f10441e3a27c40c3a70527f00c276135623a18b0e3052aecdb3" exitCode=0 Oct 30 18:35:46.694635 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.694176 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" event={"ID":"c07b8e21-48ed-49ee-8f10-135063b21802","Type":"ContainerDied","Data":"7ac70a1df0f20f10441e3a27c40c3a70527f00c276135623a18b0e3052aecdb3"} Oct 30 18:35:46.696513 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.696485 6492 generic.go:356] "Generic (PLEG): container finished" podID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerID="bfa43ba144a0103808ac0a8e1575b656307102194755d39cc5525f484087b906" exitCode=0 Oct 30 18:35:46.696645 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.696571 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" event={"ID":"acafbd00-3e35-4007-9472-261c7f8b97a9","Type":"ContainerDied","Data":"bfa43ba144a0103808ac0a8e1575b656307102194755d39cc5525f484087b906"} Oct 30 18:35:46.698815 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.698704 6492 generic.go:356] "Generic (PLEG): container finished" podID="06d212d8-ddc3-4498-af75-ac44b6553429" containerID="35eff506aa1dab41f20b150e3dde7a03d265b7ef5b056f898536534910e38915" exitCode=0 Oct 30 18:35:46.698815 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.698734 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerDied","Data":"35eff506aa1dab41f20b150e3dde7a03d265b7ef5b056f898536534910e38915"} Oct 30 18:35:46.701030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.701011 6492 generic.go:356] "Generic (PLEG): container finished" podID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerID="5edf7c99deff7042b4a488e490b297b760badb392ac5225f35af1b27a3168600" exitCode=0 Oct 30 18:35:46.701131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.701097 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" event={"ID":"df2a1ef1-0062-4369-aacb-adf384934cc9","Type":"ContainerDied","Data":"5edf7c99deff7042b4a488e490b297b760badb392ac5225f35af1b27a3168600"} Oct 30 18:35:46.703396 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.703372 6492 generic.go:356] "Generic (PLEG): container finished" podID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerID="0339df0c93cda785b1047119fa1e5b5f1ca7cf28d5f6724e4c3341ad3fc507b9" exitCode=0 Oct 30 18:35:46.703512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.703457 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" event={"ID":"d4138f84-dd0a-498c-b080-9e0b74466fad","Type":"ContainerDied","Data":"0339df0c93cda785b1047119fa1e5b5f1ca7cf28d5f6724e4c3341ad3fc507b9"} Oct 30 18:35:46.826970 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.826940 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:46.918802 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.918771 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\" (UniqueName: \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\") pod \"f059b75e-26e1-468c-9150-b6250de5e0ef\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " Oct 30 18:35:46.919005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.918835 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\") pod \"f059b75e-26e1-468c-9150-b6250de5e0ef\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " Oct 30 18:35:46.919005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.918893 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\") pod \"f059b75e-26e1-468c-9150-b6250de5e0ef\" (UID: \"f059b75e-26e1-468c-9150-b6250de5e0ef\") " Oct 30 18:35:46.919385 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.919355 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle" pod "f059b75e-26e1-468c-9150-b6250de5e0ef" (UID: "f059b75e-26e1-468c-9150-b6250de5e0ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:46.921402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.921375 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78" pod "f059b75e-26e1-468c-9150-b6250de5e0ef" (UID: "f059b75e-26e1-468c-9150-b6250de5e0ef"). InnerVolumeSpecName "kube-api-access-fbv78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:46.924555 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:46.924517 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util" pod "f059b75e-26e1-468c-9150-b6250de5e0ef" (UID: "f059b75e-26e1-468c-9150-b6250de5e0ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:46.977445 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:35:46.977404 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:35:47.020599 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.020570 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbv78\" (UniqueName: \"kubernetes.io/projected/f059b75e-26e1-468c-9150-b6250de5e0ef-kube-api-access-fbv78\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:47.020599 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.020600 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:47.020848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.020615 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f059b75e-26e1-468c-9150-b6250de5e0ef-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:47.710146 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.710106 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerStarted","Data":"54f78be36caa601c5137bb50ef8a69b59f4275767c1100b74f5d9a6024a000a4"} Oct 30 18:35:47.712260 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.712232 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" Oct 30 18:35:47.712399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.712284 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7skgth" event={"ID":"f059b75e-26e1-468c-9150-b6250de5e0ef","Type":"ContainerDied","Data":"5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85"} Oct 30 18:35:47.712399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.712321 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a17675b29c052a6854728c3ac95c89588da3196f3fabab08a7989037b563d85" Oct 30 18:35:47.870936 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.870897 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:47.920439 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.920368 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" podStartSLOduration=3.504876019 podStartE2EDuration="30.920346692s" totalImagesPullingTime="27.415470673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:47.756077652 +0000 UTC m=+219.200468648" watchObservedRunningTime="2026-10-30 18:35:47.920346692 +0000 UTC m=+219.364737687" Oct 30 18:35:47.929780 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.929750 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\") pod \"df2a1ef1-0062-4369-aacb-adf384934cc9\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " Oct 30 18:35:47.929935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.929863 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\") pod \"df2a1ef1-0062-4369-aacb-adf384934cc9\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " Oct 30 18:35:47.929935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.929919 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\" (UniqueName: \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\") pod \"df2a1ef1-0062-4369-aacb-adf384934cc9\" (UID: \"df2a1ef1-0062-4369-aacb-adf384934cc9\") " Oct 30 18:35:47.930586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.930557 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle" pod "df2a1ef1-0062-4369-aacb-adf384934cc9" (UID: "df2a1ef1-0062-4369-aacb-adf384934cc9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:47.933688 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.933658 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c" pod "df2a1ef1-0062-4369-aacb-adf384934cc9" (UID: "df2a1ef1-0062-4369-aacb-adf384934cc9"). InnerVolumeSpecName "kube-api-access-5qf4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:47.937042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.936967 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util" pod "df2a1ef1-0062-4369-aacb-adf384934cc9" (UID: "df2a1ef1-0062-4369-aacb-adf384934cc9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:47.938797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.938776 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:47.946719 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:47.946699 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:48.008617 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.008590 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:48.016666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.016636 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:48.030367 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030342 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\" (UniqueName: \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\") pod \"acafbd00-3e35-4007-9472-261c7f8b97a9\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " Oct 30 18:35:48.030561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030404 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\") pod \"acafbd00-3e35-4007-9472-261c7f8b97a9\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " Oct 30 18:35:48.030561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030514 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\") pod \"c07b8e21-48ed-49ee-8f10-135063b21802\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " Oct 30 18:35:48.030703 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030622 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\" (UniqueName: \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\") pod \"c07b8e21-48ed-49ee-8f10-135063b21802\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " Oct 30 18:35:48.030703 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030684 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\") pod \"c07b8e21-48ed-49ee-8f10-135063b21802\" (UID: \"c07b8e21-48ed-49ee-8f10-135063b21802\") " Oct 30 18:35:48.030804 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.030736 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\") pod \"acafbd00-3e35-4007-9472-261c7f8b97a9\" (UID: \"acafbd00-3e35-4007-9472-261c7f8b97a9\") " Oct 30 18:35:48.032038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.031040 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qf4c\" (UniqueName: \"kubernetes.io/projected/df2a1ef1-0062-4369-aacb-adf384934cc9-kube-api-access-5qf4c\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.032038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.031064 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.032038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.031080 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2a1ef1-0062-4369-aacb-adf384934cc9-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.032038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.031611 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle" pod "acafbd00-3e35-4007-9472-261c7f8b97a9" (UID: "acafbd00-3e35-4007-9472-261c7f8b97a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.032289 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.032228 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle" pod "c07b8e21-48ed-49ee-8f10-135063b21802" (UID: "c07b8e21-48ed-49ee-8f10-135063b21802"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.033273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.033248 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g" pod "acafbd00-3e35-4007-9472-261c7f8b97a9" (UID: "acafbd00-3e35-4007-9472-261c7f8b97a9"). InnerVolumeSpecName "kube-api-access-z2f9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:48.034057 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.034031 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2" pod "c07b8e21-48ed-49ee-8f10-135063b21802" (UID: "c07b8e21-48ed-49ee-8f10-135063b21802"). InnerVolumeSpecName "kube-api-access-5czr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:48.038970 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.038941 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util" pod "c07b8e21-48ed-49ee-8f10-135063b21802" (UID: "c07b8e21-48ed-49ee-8f10-135063b21802"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.039345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.039316 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util" pod "acafbd00-3e35-4007-9472-261c7f8b97a9" (UID: "acafbd00-3e35-4007-9472-261c7f8b97a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.131907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.131862 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\") pod \"d4138f84-dd0a-498c-b080-9e0b74466fad\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " Oct 30 18:35:48.132076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.131923 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\" (UniqueName: \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\") pod \"06d212d8-ddc3-4498-af75-ac44b6553429\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " Oct 30 18:35:48.132076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.131963 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\") pod \"d4138f84-dd0a-498c-b080-9e0b74466fad\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " Oct 30 18:35:48.132076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132023 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\") pod \"06d212d8-ddc3-4498-af75-ac44b6553429\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " Oct 30 18:35:48.132239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132139 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\") pod \"06d212d8-ddc3-4498-af75-ac44b6553429\" (UID: \"06d212d8-ddc3-4498-af75-ac44b6553429\") " Oct 30 18:35:48.132239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132225 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\" (UniqueName: \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\") pod \"d4138f84-dd0a-498c-b080-9e0b74466fad\" (UID: \"d4138f84-dd0a-498c-b080-9e0b74466fad\") " Oct 30 18:35:48.132450 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132418 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle" pod "06d212d8-ddc3-4498-af75-ac44b6553429" (UID: "06d212d8-ddc3-4498-af75-ac44b6553429"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.132606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132578 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132599 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132613 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5czr2\" (UniqueName: \"kubernetes.io/projected/c07b8e21-48ed-49ee-8f10-135063b21802-kube-api-access-5czr2\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132629 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c07b8e21-48ed-49ee-8f10-135063b21802-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132631 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle" pod "d4138f84-dd0a-498c-b080-9e0b74466fad" (UID: "d4138f84-dd0a-498c-b080-9e0b74466fad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132644 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132684 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2f9g\" (UniqueName: \"kubernetes.io/projected/acafbd00-3e35-4007-9472-261c7f8b97a9-kube-api-access-z2f9g\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.132731 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.132702 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acafbd00-3e35-4007-9472-261c7f8b97a9-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.134115 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.134095 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d" pod "06d212d8-ddc3-4498-af75-ac44b6553429" (UID: "06d212d8-ddc3-4498-af75-ac44b6553429"). InnerVolumeSpecName "kube-api-access-w242d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:48.134918 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.134887 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p" pod "d4138f84-dd0a-498c-b080-9e0b74466fad" (UID: "d4138f84-dd0a-498c-b080-9e0b74466fad"). InnerVolumeSpecName "kube-api-access-5495p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:48.139968 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.139943 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util" pod "06d212d8-ddc3-4498-af75-ac44b6553429" (UID: "06d212d8-ddc3-4498-af75-ac44b6553429"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.140205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.140186 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util" pod "d4138f84-dd0a-498c-b080-9e0b74466fad" (UID: "d4138f84-dd0a-498c-b080-9e0b74466fad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:48.234039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.233996 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.234039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.234030 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w242d\" (UniqueName: \"kubernetes.io/projected/06d212d8-ddc3-4498-af75-ac44b6553429-kube-api-access-w242d\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.234039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.234041 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4138f84-dd0a-498c-b080-9e0b74466fad-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.234039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.234053 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06d212d8-ddc3-4498-af75-ac44b6553429-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.234344 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.234061 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5495p\" (UniqueName: \"kubernetes.io/projected/d4138f84-dd0a-498c-b080-9e0b74466fad-kube-api-access-5495p\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:48.718422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.718377 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" event={"ID":"d4138f84-dd0a-498c-b080-9e0b74466fad","Type":"ContainerDied","Data":"539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc"} Oct 30 18:35:48.718422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.718419 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c884ss" Oct 30 18:35:48.718910 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.718460 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="539935b4494e14fa85f8c28e3d25c0b2a146cc8a69e25e94fd07e826ef9ddafc" Oct 30 18:35:48.720801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.720361 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" Oct 30 18:35:48.720801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.720359 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6knhr6" event={"ID":"c07b8e21-48ed-49ee-8f10-135063b21802","Type":"ContainerDied","Data":"16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78"} Oct 30 18:35:48.720801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.720485 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16174e60551fc1d30056462c4966cb2f393951e6630c5c4c6da27e4c17392c78" Oct 30 18:35:48.722493 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.722463 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" event={"ID":"acafbd00-3e35-4007-9472-261c7f8b97a9","Type":"ContainerDied","Data":"fdaaed66be79b0a7182e558d7086e5580413596726c9ed401666a871896ae828"} Oct 30 18:35:48.722650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.722515 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdaaed66be79b0a7182e558d7086e5580413596726c9ed401666a871896ae828" Oct 30 18:35:48.722650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.722486 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71lscbl" Oct 30 18:35:48.724526 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.724507 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" Oct 30 18:35:48.724526 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.724518 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gmc4g" event={"ID":"06d212d8-ddc3-4498-af75-ac44b6553429","Type":"ContainerDied","Data":"a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087"} Oct 30 18:35:48.724678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.724569 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a25bfbc4438990a0870bea1ef8a822e97f00b54de64649456ef819f1f85e1087" Oct 30 18:35:48.726853 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.726830 6492 generic.go:356] "Generic (PLEG): container finished" podID="454162e5-cc43-4798-8490-2efcc954f558" containerID="54f78be36caa601c5137bb50ef8a69b59f4275767c1100b74f5d9a6024a000a4" exitCode=0 Oct 30 18:35:48.726956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.726891 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerDied","Data":"54f78be36caa601c5137bb50ef8a69b59f4275767c1100b74f5d9a6024a000a4"} Oct 30 18:35:48.728621 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.728606 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" Oct 30 18:35:48.728706 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.728602 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axmjpc" event={"ID":"df2a1ef1-0062-4369-aacb-adf384934cc9","Type":"ContainerDied","Data":"0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c"} Oct 30 18:35:48.728763 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:48.728732 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0caff912cad62d5e79231cd57c62e26a7e3ec6884398744418b19f26acfdb24c" Oct 30 18:35:49.856252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.856223 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:49.948622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.948584 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\") pod \"454162e5-cc43-4798-8490-2efcc954f558\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " Oct 30 18:35:49.948801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.948684 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\" (UniqueName: \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\") pod \"454162e5-cc43-4798-8490-2efcc954f558\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " Oct 30 18:35:49.948801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.948717 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\") pod \"454162e5-cc43-4798-8490-2efcc954f558\" (UID: \"454162e5-cc43-4798-8490-2efcc954f558\") " Oct 30 18:35:49.949364 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.949335 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle" pod "454162e5-cc43-4798-8490-2efcc954f558" (UID: "454162e5-cc43-4798-8490-2efcc954f558"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:49.951042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.951009 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq" pod "454162e5-cc43-4798-8490-2efcc954f558" (UID: "454162e5-cc43-4798-8490-2efcc954f558"). InnerVolumeSpecName "kube-api-access-mfptq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:49.954373 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:49.954344 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util" pod "454162e5-cc43-4798-8490-2efcc954f558" (UID: "454162e5-cc43-4798-8490-2efcc954f558"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:50.049959 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.049867 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfptq\" (UniqueName: \"kubernetes.io/projected/454162e5-cc43-4798-8490-2efcc954f558-kube-api-access-mfptq\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:50.049959 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.049902 6492 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-bundle\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:50.049959 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.049915 6492 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/454162e5-cc43-4798-8490-2efcc954f558-util\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:35:50.739285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.739244 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" event={"ID":"454162e5-cc43-4798-8490-2efcc954f558","Type":"ContainerDied","Data":"b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921"} Oct 30 18:35:50.739285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.739269 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c75m8m" Oct 30 18:35:50.739285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:50.739292 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b246d86ea7f77774bc91dc9e5c613c094786f8ab4f3d5dddcd65b094f6c3b921" Oct 30 18:35:53.130254 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130219 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl"] Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130529 6492 cpu_manager.go:414] "containerMap: removing container" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130555 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130568 6492 cpu_manager.go:414] "containerMap: removing container" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130573 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130580 6492 cpu_manager.go:414] "containerMap: removing container" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130585 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130592 6492 cpu_manager.go:414] "containerMap: removing container" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130596 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130603 6492 cpu_manager.go:414] "containerMap: removing container" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130607 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130613 6492 cpu_manager.go:414] "containerMap: removing container" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130618 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130626 6492 cpu_manager.go:414] "containerMap: removing container" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130630 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130637 6492 cpu_manager.go:414] "containerMap: removing container" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130641 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130647 6492 cpu_manager.go:414] "containerMap: removing container" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130652 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130664 6492 cpu_manager.go:414] "containerMap: removing container" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130671 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130677 6492 cpu_manager.go:414] "containerMap: removing container" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130681 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130688 6492 cpu_manager.go:414] "containerMap: removing container" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130694 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130701 6492 cpu_manager.go:414] "containerMap: removing container" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130705 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="pull" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130711 6492 cpu_manager.go:414] "containerMap: removing container" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130716 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130721 6492 cpu_manager.go:414] "containerMap: removing container" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130725 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="util" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130731 6492 cpu_manager.go:414] "containerMap: removing container" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130735 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130741 6492 cpu_manager.go:414] "containerMap: removing container" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="extract" Oct 30 18:35:53.130730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130746 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130755 6492 cpu_manager.go:414] "containerMap: removing container" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130759 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130767 6492 cpu_manager.go:414] "containerMap: removing container" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="util" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130771 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="util" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130780 6492 cpu_manager.go:414] "containerMap: removing container" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130784 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130796 6492 cpu_manager.go:414] "containerMap: removing container" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130802 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="pull" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130856 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4138f84-dd0a-498c-b080-9e0b74466fad" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130863 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="c07b8e21-48ed-49ee-8f10-135063b21802" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130870 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="454162e5-cc43-4798-8490-2efcc954f558" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130875 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="df2a1ef1-0062-4369-aacb-adf384934cc9" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130880 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="acafbd00-3e35-4007-9472-261c7f8b97a9" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130887 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="06d212d8-ddc3-4498-af75-ac44b6553429" containerName="extract" Oct 30 18:35:53.131772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.130892 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="f059b75e-26e1-468c-9150-b6250de5e0ef" containerName="extract" Oct 30 18:35:53.147504 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.147479 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.158803 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.158778 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl"] Oct 30 18:35:53.170038 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.170013 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Oct 30 18:35:53.170182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.170068 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:35:53.170259 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.170210 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-r6tdk\"" Oct 30 18:35:53.277688 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.277651 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a858dd87-eee3-404c-8113-a8d745f50304-tmp\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.277863 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.277698 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbvwm\" (UniqueName: \"kubernetes.io/projected/a858dd87-eee3-404c-8113-a8d745f50304-kube-api-access-cbvwm\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.378750 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.378720 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a858dd87-eee3-404c-8113-a8d745f50304-tmp\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.378919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.378768 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbvwm\" (UniqueName: \"kubernetes.io/projected/a858dd87-eee3-404c-8113-a8d745f50304-kube-api-access-cbvwm\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.379168 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.379145 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a858dd87-eee3-404c-8113-a8d745f50304-tmp\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.409506 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.409435 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbvwm\" (UniqueName: \"kubernetes.io/projected/a858dd87-eee3-404c-8113-a8d745f50304-kube-api-access-cbvwm\") pod \"jobset-operator-674cd59dc5-mfkvl\" (UID: \"a858dd87-eee3-404c-8113-a8d745f50304\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.482018 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.481981 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" Oct 30 18:35:53.802416 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:53.802382 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl"] Oct 30 18:35:53.805314 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:53.805287 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda858dd87_eee3_404c_8113_a8d745f50304.slice/crio-fae917f8615dae945b468df1ee9acd7b46bb929a13d9953a369fb5658444a312 WatchSource:0}: Error finding container fae917f8615dae945b468df1ee9acd7b46bb929a13d9953a369fb5658444a312: Status 404 returned error can't find the container with id fae917f8615dae945b468df1ee9acd7b46bb929a13d9953a369fb5658444a312 Oct 30 18:35:54.758693 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:54.758657 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" event={"ID":"a858dd87-eee3-404c-8113-a8d745f50304","Type":"ContainerStarted","Data":"fae917f8615dae945b468df1ee9acd7b46bb929a13d9953a369fb5658444a312"} Oct 30 18:35:55.764787 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:55.764748 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" event={"ID":"a858dd87-eee3-404c-8113-a8d745f50304","Type":"ContainerStarted","Data":"dd79361a2efa026afeee2e5822808dfc2762fb8f2288248ac45a22e68dce97fb"} Oct 30 18:35:55.808942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:55.808883 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-mfkvl" podStartSLOduration=1.009787574 podStartE2EDuration="2.808870523s" totalImagesPullingTime="1.799082949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:55.807366674 +0000 UTC m=+227.251757690" watchObservedRunningTime="2026-10-30 18:35:55.808870523 +0000 UTC m=+227.253261518" Oct 30 18:35:58.866740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.866701 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5"] Oct 30 18:35:58.870017 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.870001 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:58.903688 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.903658 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Oct 30 18:35:58.904243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.904227 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Oct 30 18:35:58.936641 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.936601 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5"] Oct 30 18:35:58.949771 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.949741 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 18:35:58.950226 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.950205 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Oct 30 18:35:58.950226 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.950217 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 18:35:58.950226 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:58.950211 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-znpc4\"" Oct 30 18:35:59.036711 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036673 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.036711 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036712 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.036976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036811 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgr8\" (UniqueName: \"kubernetes.io/projected/49d576d5-c54d-4b82-bdfa-55610d101434-kube-api-access-wsgr8\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.036976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036853 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.036976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036927 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.036976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.036971 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138255 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138162 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138255 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138213 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138464 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138392 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgr8\" (UniqueName: \"kubernetes.io/projected/49d576d5-c54d-4b82-bdfa-55610d101434-kube-api-access-wsgr8\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138464 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138450 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138484 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138519 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.138912 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.138882 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.140590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.140566 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d576d5-c54d-4b82-bdfa-55610d101434-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.140848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.140829 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.140915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.140886 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.140975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.140914 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49d576d5-c54d-4b82-bdfa-55610d101434-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.169798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.169773 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgr8\" (UniqueName: \"kubernetes.io/projected/49d576d5-c54d-4b82-bdfa-55610d101434-kube-api-access-wsgr8\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5\" (UID: \"49d576d5-c54d-4b82-bdfa-55610d101434\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.184682 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.184658 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:35:59.571262 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.571237 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5"] Oct 30 18:35:59.572332 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:35:59.572307 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d576d5_c54d_4b82_bdfa_55610d101434.slice/crio-d580d7e51e89be557d459f236ae2305c3996b147cd99cc4ceede0d93bfa28119 WatchSource:0}: Error finding container d580d7e51e89be557d459f236ae2305c3996b147cd99cc4ceede0d93bfa28119: Status 404 returned error can't find the container with id d580d7e51e89be557d459f236ae2305c3996b147cd99cc4ceede0d93bfa28119 Oct 30 18:35:59.783809 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:35:59.783772 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" event={"ID":"49d576d5-c54d-4b82-bdfa-55610d101434","Type":"ContainerStarted","Data":"d580d7e51e89be557d459f236ae2305c3996b147cd99cc4ceede0d93bfa28119"} Oct 30 18:36:02.802754 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:02.802706 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" event={"ID":"49d576d5-c54d-4b82-bdfa-55610d101434","Type":"ContainerStarted","Data":"5a28adc09cf788be9d1050661102ac3523d6b7148d81a7365e3345d1ba092ece"} Oct 30 18:36:03.808573 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:03.808525 6492 generic.go:356] "Generic (PLEG): container finished" podID="49d576d5-c54d-4b82-bdfa-55610d101434" containerID="5a28adc09cf788be9d1050661102ac3523d6b7148d81a7365e3345d1ba092ece" exitCode=0 Oct 30 18:36:03.808975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:03.808580 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" event={"ID":"49d576d5-c54d-4b82-bdfa-55610d101434","Type":"ContainerDied","Data":"5a28adc09cf788be9d1050661102ac3523d6b7148d81a7365e3345d1ba092ece"} Oct 30 18:36:04.814335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:04.814303 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" event={"ID":"49d576d5-c54d-4b82-bdfa-55610d101434","Type":"ContainerStarted","Data":"c7df5cc6af2db993fa2e545328a6312508c4f0a0d00a2cdcefb75618a84a4586"} Oct 30 18:36:04.814795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:04.814459 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:36:04.903795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:04.903742 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" podStartSLOduration=3.236182871 podStartE2EDuration="6.903729328s" totalImagesPullingTime="3.211392779s" totalInitContainerRuntime="456.153678ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:04.891138175 +0000 UTC m=+236.335529181" watchObservedRunningTime="2026-10-30 18:36:04.903729328 +0000 UTC m=+236.348120323" Oct 30 18:36:15.821451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:15.821417 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5" Oct 30 18:36:16.971780 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:36:16.971742 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:36:17.052435 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.052398 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv"] Oct 30 18:36:17.058988 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.058963 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.077197 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.077170 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv"] Oct 30 18:36:17.088347 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.088319 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:17.088347 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.088343 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qkdvk\"" Oct 30 18:36:17.088501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.088319 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:17.089705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.089684 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.089752 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.089716 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87h8w\" (UniqueName: \"kubernetes.io/projected/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-kube-api-access-87h8w\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.190593 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.190558 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.190755 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.190601 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87h8w\" (UniqueName: \"kubernetes.io/projected/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-kube-api-access-87h8w\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.190954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.190934 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.221970 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.221896 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87h8w\" (UniqueName: \"kubernetes.io/projected/03ed68d6-bb95-4dbf-9f07-c95a7fe49c69-kube-api-access-87h8w\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv\" (UID: \"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.255564 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.255520 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6"] Oct 30 18:36:17.258991 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.258974 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.279140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.279115 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x695g\"" Oct 30 18:36:17.279140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.279124 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 18:36:17.279338 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.279322 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 18:36:17.287406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.287381 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6"] Oct 30 18:36:17.291325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.291303 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2xv\" (UniqueName: \"kubernetes.io/projected/194f4456-d1cb-449d-8b4b-66301f152cf7-kube-api-access-tx2xv\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.291433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.291353 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/194f4456-d1cb-449d-8b4b-66301f152cf7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.371050 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.371022 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" Oct 30 18:36:17.392408 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.392381 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2xv\" (UniqueName: \"kubernetes.io/projected/194f4456-d1cb-449d-8b4b-66301f152cf7-kube-api-access-tx2xv\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.392527 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.392420 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/194f4456-d1cb-449d-8b4b-66301f152cf7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.392772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.392755 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/194f4456-d1cb-449d-8b4b-66301f152cf7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.417701 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.417672 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2xv\" (UniqueName: \"kubernetes.io/projected/194f4456-d1cb-449d-8b4b-66301f152cf7-kube-api-access-tx2xv\") pod \"kuadrant-operator-controller-manager-678d87766f-fr7x6\" (UID: \"194f4456-d1cb-449d-8b4b-66301f152cf7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.571851 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.571752 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:17.700302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.700269 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv"] Oct 30 18:36:17.703693 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:36:17.703665 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ed68d6_bb95_4dbf_9f07_c95a7fe49c69.slice/crio-babf4b283e2489c0c9b6deb2858b47ba0c4b4dbce4f5589c7eecc925da4e81d5 WatchSource:0}: Error finding container babf4b283e2489c0c9b6deb2858b47ba0c4b4dbce4f5589c7eecc925da4e81d5: Status 404 returned error can't find the container with id babf4b283e2489c0c9b6deb2858b47ba0c4b4dbce4f5589c7eecc925da4e81d5 Oct 30 18:36:17.870054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.869972 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" event={"ID":"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69","Type":"ContainerStarted","Data":"babf4b283e2489c0c9b6deb2858b47ba0c4b4dbce4f5589c7eecc925da4e81d5"} Oct 30 18:36:17.883309 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:17.883250 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6"] Oct 30 18:36:18.876067 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:18.876028 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" event={"ID":"194f4456-d1cb-449d-8b4b-66301f152cf7","Type":"ContainerStarted","Data":"e0b7ea9599c09925db5d5585aa7dffb383d02e557f31cb81864c247f4062d849"} Oct 30 18:36:19.883940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:19.883899 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" event={"ID":"03ed68d6-bb95-4dbf-9f07-c95a7fe49c69","Type":"ContainerStarted","Data":"e3e3044b83c5a3d7b853d1a42f7616ebce9d4f302b51103daf0cd7c8c3a597e2"} Oct 30 18:36:19.964047 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:19.963976 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vmxxv" podStartSLOduration=2.432848983 podStartE2EDuration="3.963922542s" totalImagesPullingTime="1.531073559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:19.950390621 +0000 UTC m=+251.394781617" watchObservedRunningTime="2026-10-30 18:36:19.963922542 +0000 UTC m=+251.408313540" Oct 30 18:36:22.530256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.530222 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh"] Oct 30 18:36:22.533688 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.533660 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.560930 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.560898 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 18:36:22.572992 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.572963 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 18:36:22.572992 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.572963 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-pbf8n\"" Oct 30 18:36:22.575236 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.575214 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh"] Oct 30 18:36:22.640458 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.640420 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.640694 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.640480 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8cnp\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-kube-api-access-p8cnp\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.741957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.741898 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8cnp\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-kube-api-access-p8cnp\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.742146 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.742002 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.770113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.770081 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8cnp\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-kube-api-access-p8cnp\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.770256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.770151 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d507704-680c-4354-9395-da664ac20a6b-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-rhkgh\" (UID: \"4d507704-680c-4354-9395-da664ac20a6b\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.846451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.846345 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:22.904779 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.904744 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" event={"ID":"194f4456-d1cb-449d-8b4b-66301f152cf7","Type":"ContainerStarted","Data":"de93c320217bdaec6a44a1358e925fffb265523fc08ad1ee70181a05a498a212"} Oct 30 18:36:22.904910 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.904849 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:22.949886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:22.949834 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" podStartSLOduration=1.6717318620000001 podStartE2EDuration="5.949819525s" totalImagesPullingTime="4.278087663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:22.948629739 +0000 UTC m=+254.393020747" watchObservedRunningTime="2026-10-30 18:36:22.949819525 +0000 UTC m=+254.394210520" Oct 30 18:36:23.166364 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:23.166332 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh"] Oct 30 18:36:23.911552 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:23.911512 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" event={"ID":"4d507704-680c-4354-9395-da664ac20a6b","Type":"ContainerStarted","Data":"44d82b100cad548b369a215a09d3692a4337f3a7308a03884172837125b9e349"} Oct 30 18:36:26.927181 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:26.927144 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" event={"ID":"4d507704-680c-4354-9395-da664ac20a6b","Type":"ContainerStarted","Data":"1a48358e8d69ca4b653ec855ed8f022b3b4dbb277721a031b13f9763200db9fa"} Oct 30 18:36:26.927691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:26.927282 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:26.971304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:26.971253 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" podStartSLOduration=2.276987905 podStartE2EDuration="4.97123863s" totalImagesPullingTime="2.694250725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:26.970273477 +0000 UTC m=+258.414664473" watchObservedRunningTime="2026-10-30 18:36:26.97123863 +0000 UTC m=+258.415629625" Oct 30 18:36:28.056599 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.056528 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-mkwbn"] Oct 30 18:36:28.059897 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.059880 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.088489 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.088458 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 30 18:36:28.088705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.088505 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qx7lj\"" Oct 30 18:36:28.088705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.088575 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 30 18:36:28.089618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.089593 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-mkwbn"] Oct 30 18:36:28.191195 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.191151 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzm4s\" (UniqueName: \"kubernetes.io/projected/7acc71a4-3041-403f-a759-707008803428-kube-api-access-lzm4s\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.191382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.191203 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acc71a4-3041-403f-a759-707008803428-tls-certs\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.191382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.191221 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7acc71a4-3041-403f-a759-707008803428-tmp\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.292268 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.292234 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzm4s\" (UniqueName: \"kubernetes.io/projected/7acc71a4-3041-403f-a759-707008803428-kube-api-access-lzm4s\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.292447 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.292372 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acc71a4-3041-403f-a759-707008803428-tls-certs\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.292447 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.292406 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7acc71a4-3041-403f-a759-707008803428-tmp\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.295044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.295015 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7acc71a4-3041-403f-a759-707008803428-tmp\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.295165 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.295051 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7acc71a4-3041-403f-a759-707008803428-tls-certs\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.356239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.356172 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzm4s\" (UniqueName: \"kubernetes.io/projected/7acc71a4-3041-403f-a759-707008803428-kube-api-access-lzm4s\") pod \"kube-auth-proxy-869444d94f-mkwbn\" (UID: \"7acc71a4-3041-403f-a759-707008803428\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.372006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.371970 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" Oct 30 18:36:28.685480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.685445 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-mkwbn"] Oct 30 18:36:28.687449 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:36:28.687419 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acc71a4_3041_403f_a759_707008803428.slice/crio-f1ca28adcd04409ce0b204c3d0ad72f93f45dc5680db392f473676394dae9866 WatchSource:0}: Error finding container f1ca28adcd04409ce0b204c3d0ad72f93f45dc5680db392f473676394dae9866: Status 404 returned error can't find the container with id f1ca28adcd04409ce0b204c3d0ad72f93f45dc5680db392f473676394dae9866 Oct 30 18:36:28.943268 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:28.943188 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" event={"ID":"7acc71a4-3041-403f-a759-707008803428","Type":"ContainerStarted","Data":"f1ca28adcd04409ce0b204c3d0ad72f93f45dc5680db392f473676394dae9866"} Oct 30 18:36:29.212618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.212582 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:29.217114 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.217092 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:29.241159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.241129 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tp2f5\"" Oct 30 18:36:29.243327 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.243296 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:29.303963 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.303911 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qtd\" (UniqueName: \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\") pod \"authorino-57f7f8659f-ljl9r\" (UID: \"1888e090-ed9f-41b0-97aa-67a1bec31dd0\") " pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:29.359971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.359936 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:36:29.363585 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.363532 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:36:29.401580 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.401520 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:36:29.404964 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.404936 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qtd\" (UniqueName: \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\") pod \"authorino-57f7f8659f-ljl9r\" (UID: \"1888e090-ed9f-41b0-97aa-67a1bec31dd0\") " pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:29.446804 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.446771 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qtd\" (UniqueName: \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\") pod \"authorino-57f7f8659f-ljl9r\" (UID: \"1888e090-ed9f-41b0-97aa-67a1bec31dd0\") " pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:29.505902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.505807 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ffz\" (UniqueName: \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\") pod \"authorino-56c6c9ddd5-jfhw9\" (UID: \"9485a25c-0712-49a2-b47f-081392ab0cc7\") " pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:36:29.528712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.528674 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:29.610203 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.607219 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ffz\" (UniqueName: \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\") pod \"authorino-56c6c9ddd5-jfhw9\" (UID: \"9485a25c-0712-49a2-b47f-081392ab0cc7\") " pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:36:29.632582 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.632529 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ffz\" (UniqueName: \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\") pod \"authorino-56c6c9ddd5-jfhw9\" (UID: \"9485a25c-0712-49a2-b47f-081392ab0cc7\") " pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:36:29.680623 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.680349 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:36:29.859762 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.859644 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:29.864011 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:36:29.863979 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1888e090_ed9f_41b0_97aa_67a1bec31dd0.slice/crio-3c562ba34c91b4d034c4a0d241f7a85850f0bc6e8ef7b8ad881ae60c472a1de1 WatchSource:0}: Error finding container 3c562ba34c91b4d034c4a0d241f7a85850f0bc6e8ef7b8ad881ae60c472a1de1: Status 404 returned error can't find the container with id 3c562ba34c91b4d034c4a0d241f7a85850f0bc6e8ef7b8ad881ae60c472a1de1 Oct 30 18:36:29.950619 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.950580 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" event={"ID":"1888e090-ed9f-41b0-97aa-67a1bec31dd0","Type":"ContainerStarted","Data":"3c562ba34c91b4d034c4a0d241f7a85850f0bc6e8ef7b8ad881ae60c472a1de1"} Oct 30 18:36:29.999667 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:29.999487 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:36:30.002339 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:36:30.002305 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9485a25c_0712_49a2_b47f_081392ab0cc7.slice/crio-fa74b2b186239e1f01b5d29529a3e549e9945b002d44f0c7177b77c142bb549e WatchSource:0}: Error finding container fa74b2b186239e1f01b5d29529a3e549e9945b002d44f0c7177b77c142bb549e: Status 404 returned error can't find the container with id fa74b2b186239e1f01b5d29529a3e549e9945b002d44f0c7177b77c142bb549e Oct 30 18:36:30.959784 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:30.959741 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" event={"ID":"9485a25c-0712-49a2-b47f-081392ab0cc7","Type":"ContainerStarted","Data":"fa74b2b186239e1f01b5d29529a3e549e9945b002d44f0c7177b77c142bb549e"} Oct 30 18:36:31.968430 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:31.968377 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" event={"ID":"7acc71a4-3041-403f-a759-707008803428","Type":"ContainerStarted","Data":"5827ee2923e5b95b97ee4c148969a571911911cbd5f66df7886911e380e81557"} Oct 30 18:36:32.070972 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:32.070899 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-869444d94f-mkwbn" podStartSLOduration=1.66764946 podStartE2EDuration="4.070878789s" totalImagesPullingTime="2.403229329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:32.069643835 +0000 UTC m=+263.514034859" watchObservedRunningTime="2026-10-30 18:36:32.070878789 +0000 UTC m=+263.515269786" Oct 30 18:36:32.935076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:32.934992 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-rhkgh" Oct 30 18:36:32.977473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:32.977439 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" event={"ID":"1888e090-ed9f-41b0-97aa-67a1bec31dd0","Type":"ContainerStarted","Data":"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39"} Oct 30 18:36:32.984197 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:32.984156 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" event={"ID":"9485a25c-0712-49a2-b47f-081392ab0cc7","Type":"ContainerStarted","Data":"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776"} Oct 30 18:36:33.034878 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:33.034816 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" podStartSLOduration=1.346096961 podStartE2EDuration="4.034798415s" totalImagesPullingTime="2.688701454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:33.032424812 +0000 UTC m=+264.476815802" watchObservedRunningTime="2026-10-30 18:36:33.034798415 +0000 UTC m=+264.479189413" Oct 30 18:36:33.078496 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:33.078444 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" podStartSLOduration=1.523909809 podStartE2EDuration="4.078428996s" totalImagesPullingTime="2.554519187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:33.077404192 +0000 UTC m=+264.521795187" watchObservedRunningTime="2026-10-30 18:36:33.078428996 +0000 UTC m=+264.522819993" Oct 30 18:36:33.131954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:33.131922 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:33.913650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:33.913620 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fr7x6" Oct 30 18:36:34.995705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:34.995659 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" podUID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" containerName="authorino" containerID="cri-o://392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39" gracePeriod=30 Oct 30 18:36:35.230798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:35.230771 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:35.364370 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:35.364276 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\" (UniqueName: \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\") pod \"1888e090-ed9f-41b0-97aa-67a1bec31dd0\" (UID: \"1888e090-ed9f-41b0-97aa-67a1bec31dd0\") " Oct 30 18:36:35.366699 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:35.366666 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd" pod "1888e090-ed9f-41b0-97aa-67a1bec31dd0" (UID: "1888e090-ed9f-41b0-97aa-67a1bec31dd0"). InnerVolumeSpecName "kube-api-access-g2qtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:36:35.465636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:35.465591 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2qtd\" (UniqueName: \"kubernetes.io/projected/1888e090-ed9f-41b0-97aa-67a1bec31dd0-kube-api-access-g2qtd\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:36:36.001519 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.001484 6492 generic.go:356] "Generic (PLEG): container finished" podID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" containerID="392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39" exitCode=0 Oct 30 18:36:36.001942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.001528 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" Oct 30 18:36:36.001942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.001528 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" event={"ID":"1888e090-ed9f-41b0-97aa-67a1bec31dd0","Type":"ContainerDied","Data":"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39"} Oct 30 18:36:36.001942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.001663 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-ljl9r" event={"ID":"1888e090-ed9f-41b0-97aa-67a1bec31dd0","Type":"ContainerDied","Data":"3c562ba34c91b4d034c4a0d241f7a85850f0bc6e8ef7b8ad881ae60c472a1de1"} Oct 30 18:36:36.001942 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.001687 6492 scope.go:122] "RemoveContainer" containerID="392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39" Oct 30 18:36:36.010720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.010705 6492 scope.go:122] "RemoveContainer" containerID="392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39" Oct 30 18:36:36.010974 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:36:36.010954 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39\": container with ID starting with 392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39 not found: ID does not exist" containerID="392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39" Oct 30 18:36:36.011039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.010986 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39"} err="failed to get container status \"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39\": rpc error: code = NotFound desc = could not find container \"392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39\": container with ID starting with 392b349d0bbc93b7e121b3a5bb2214832bceb9d7ee1ba5133aa236395ab59e39 not found: ID does not exist" Oct 30 18:36:36.061611 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.061574 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:36.075456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.075426 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-ljl9r"] Oct 30 18:36:36.663448 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:36.663405 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" path="/var/lib/kubelet/pods/1888e090-ed9f-41b0-97aa-67a1bec31dd0/volumes" Oct 30 18:36:38.028781 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.028742 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg"] Oct 30 18:36:38.029157 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.029116 6492 cpu_manager.go:414] "containerMap: removing container" podUID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" containerName="authorino" Oct 30 18:36:38.029157 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.029129 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" containerName="authorino" Oct 30 18:36:38.029239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.029192 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="1888e090-ed9f-41b0-97aa-67a1bec31dd0" containerName="authorino" Oct 30 18:36:38.032841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.032820 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.065697 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.065667 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 18:36:38.076371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.076333 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 18:36:38.101445 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.101415 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-pbc2c\"" Oct 30 18:36:38.101667 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.101513 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Oct 30 18:36:38.103271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.103242 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg"] Oct 30 18:36:38.192013 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.191973 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.192187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.192023 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01ee88b-5ace-448e-b749-0047a0b60a7e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.192187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.192129 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.192256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.192194 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vjn6\" (UniqueName: \"kubernetes.io/projected/c01ee88b-5ace-448e-b749-0047a0b60a7e-kube-api-access-5vjn6\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293309 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vjn6\" (UniqueName: \"kubernetes.io/projected/c01ee88b-5ace-448e-b749-0047a0b60a7e-kube-api-access-5vjn6\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293362 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293530 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01ee88b-5ace-448e-b749-0047a0b60a7e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293657 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293626 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293872 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293850 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.293973 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.293955 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c01ee88b-5ace-448e-b749-0047a0b60a7e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.294353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.294336 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01ee88b-5ace-448e-b749-0047a0b60a7e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.316690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.316656 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vjn6\" (UniqueName: \"kubernetes.io/projected/c01ee88b-5ace-448e-b749-0047a0b60a7e-kube-api-access-5vjn6\") pod \"kserve-module-controller-manager-7566cf5fc8-l8dsg\" (UID: \"c01ee88b-5ace-448e-b749-0047a0b60a7e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.345141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.345090 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:38.664474 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:36:38.664441 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01ee88b_5ace_448e_b749_0047a0b60a7e.slice/crio-f80576e782392f1ad3a24e85ef289c44a4bfaca324e755751e04153a6c9a93c1 WatchSource:0}: Error finding container f80576e782392f1ad3a24e85ef289c44a4bfaca324e755751e04153a6c9a93c1: Status 404 returned error can't find the container with id f80576e782392f1ad3a24e85ef289c44a4bfaca324e755751e04153a6c9a93c1 Oct 30 18:36:38.665078 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:38.665057 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg"] Oct 30 18:36:39.019033 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:39.018997 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" event={"ID":"c01ee88b-5ace-448e-b749-0047a0b60a7e","Type":"ContainerStarted","Data":"f80576e782392f1ad3a24e85ef289c44a4bfaca324e755751e04153a6c9a93c1"} Oct 30 18:36:41.038062 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:41.038017 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" event={"ID":"c01ee88b-5ace-448e-b749-0047a0b60a7e","Type":"ContainerStarted","Data":"c388728173fd71a90cdc6263dcf2d0fc66435b7d0716ed91816da7de5c3a2ca6"} Oct 30 18:36:41.038470 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:41.038195 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:36:41.098312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:36:41.098253 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" podStartSLOduration=2.571093591 podStartE2EDuration="4.098238671s" totalImagesPullingTime="1.52714508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:41.097902616 +0000 UTC m=+272.542293631" watchObservedRunningTime="2026-10-30 18:36:41.098238671 +0000 UTC m=+272.542629712" Oct 30 18:36:46.972478 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:36:46.972432 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:37:02.044010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:02.043925 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-l8dsg" Oct 30 18:37:08.706881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:08.706854 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:37:08.707220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:08.706861 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:37:11.325585 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.325553 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:11.332708 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.332688 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.350336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.350315 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 30 18:37:11.350336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.350316 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 30 18:37:11.361990 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.361969 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4q6fz\"" Oct 30 18:37:11.362092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.362057 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 30 18:37:11.363625 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.363604 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:11.366830 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.366812 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsqxz\" (UniqueName: \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.366893 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.366860 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.367016 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.366998 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.467621 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.467587 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.467797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.467634 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsqxz\" (UniqueName: \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.467797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.467676 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.470134 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.470103 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.470251 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.470133 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.491069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.491041 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsqxz\" (UniqueName: \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\") pod \"maas-controller-56d4bb9f9f-9pwh4\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.644457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.644371 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:11.947802 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:11.947770 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:11.949566 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:11.949527 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ee2912_a7f7_4149_99e4_a53f2379003e.slice/crio-c5b09408f950f4f2f6b530da5bb589d2679f4a7a4f20e51784c5b04445b53eb7 WatchSource:0}: Error finding container c5b09408f950f4f2f6b530da5bb589d2679f4a7a4f20e51784c5b04445b53eb7: Status 404 returned error can't find the container with id c5b09408f950f4f2f6b530da5bb589d2679f4a7a4f20e51784c5b04445b53eb7 Oct 30 18:37:12.181826 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:12.181793 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" event={"ID":"30ee2912-a7f7-4149-99e4-a53f2379003e","Type":"ContainerStarted","Data":"c5b09408f950f4f2f6b530da5bb589d2679f4a7a4f20e51784c5b04445b53eb7"} Oct 30 18:37:15.198723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:15.198674 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" event={"ID":"30ee2912-a7f7-4149-99e4-a53f2379003e","Type":"ContainerStarted","Data":"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e"} Oct 30 18:37:15.199209 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:15.198938 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:15.242154 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:15.242094 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" podStartSLOduration=1.949944187 podStartE2EDuration="4.242080361s" totalImagesPullingTime="2.292136174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:15.240827632 +0000 UTC m=+306.685218627" watchObservedRunningTime="2026-10-30 18:37:15.242080361 +0000 UTC m=+306.686471357" Oct 30 18:37:16.973147 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:37:16.973108 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:37:25.912861 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:25.912814 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:25.913375 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:25.913130 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" podUID="30ee2912-a7f7-4149-99e4-a53f2379003e" containerName="manager" containerID="cri-o://aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e" gracePeriod=30 Oct 30 18:37:25.914514 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:25.914488 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:26.159842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.159817 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:26.276150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.276115 6492 generic.go:356] "Generic (PLEG): container finished" podID="30ee2912-a7f7-4149-99e4-a53f2379003e" containerID="aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e" exitCode=0 Oct 30 18:37:26.276355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.276179 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" Oct 30 18:37:26.276355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.276211 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" event={"ID":"30ee2912-a7f7-4149-99e4-a53f2379003e","Type":"ContainerDied","Data":"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e"} Oct 30 18:37:26.276355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.276278 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d4bb9f9f-9pwh4" event={"ID":"30ee2912-a7f7-4149-99e4-a53f2379003e","Type":"ContainerDied","Data":"c5b09408f950f4f2f6b530da5bb589d2679f4a7a4f20e51784c5b04445b53eb7"} Oct 30 18:37:26.276355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.276309 6492 scope.go:122] "RemoveContainer" containerID="aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e" Oct 30 18:37:26.287257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.287236 6492 scope.go:122] "RemoveContainer" containerID="aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e" Oct 30 18:37:26.287639 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:37:26.287615 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e\": container with ID starting with aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e not found: ID does not exist" containerID="aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e" Oct 30 18:37:26.287729 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.287652 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e"} err="failed to get container status \"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e\": rpc error: code = NotFound desc = could not find container \"aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e\": container with ID starting with aa77f631c4ac25332ac67bc4818e06345310531f3dd78bfb2aab87a4e3de791e not found: ID does not exist" Oct 30 18:37:26.316681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.316645 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\") pod \"30ee2912-a7f7-4149-99e4-a53f2379003e\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " Oct 30 18:37:26.316874 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.316711 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\" (UniqueName: \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\") pod \"30ee2912-a7f7-4149-99e4-a53f2379003e\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " Oct 30 18:37:26.316874 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.316783 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\") pod \"30ee2912-a7f7-4149-99e4-a53f2379003e\" (UID: \"30ee2912-a7f7-4149-99e4-a53f2379003e\") " Oct 30 18:37:26.318975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.318936 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert" pod "30ee2912-a7f7-4149-99e4-a53f2379003e" (UID: "30ee2912-a7f7-4149-99e4-a53f2379003e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:37:26.319102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.319038 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz" pod "30ee2912-a7f7-4149-99e4-a53f2379003e" (UID: "30ee2912-a7f7-4149-99e4-a53f2379003e"). InnerVolumeSpecName "kube-api-access-zsqxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:37:26.319102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.319081 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs" pod "30ee2912-a7f7-4149-99e4-a53f2379003e" (UID: "30ee2912-a7f7-4149-99e4-a53f2379003e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:37:26.417795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.417751 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsqxz\" (UniqueName: \"kubernetes.io/projected/30ee2912-a7f7-4149-99e4-a53f2379003e-kube-api-access-zsqxz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:37:26.417795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.417791 6492 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-metrics-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:37:26.417795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.417800 6492 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30ee2912-a7f7-4149-99e4-a53f2379003e-webhook-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:37:26.625256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.625222 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:26.637155 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.637127 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56d4bb9f9f-9pwh4"] Oct 30 18:37:26.663979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:26.663943 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30ee2912-a7f7-4149-99e4-a53f2379003e" path="/var/lib/kubelet/pods/30ee2912-a7f7-4149-99e4-a53f2379003e/volumes" Oct 30 18:37:35.788009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.787975 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-s47vd"] Oct 30 18:37:35.788518 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.788354 6492 cpu_manager.go:414] "containerMap: removing container" podUID="30ee2912-a7f7-4149-99e4-a53f2379003e" containerName="manager" Oct 30 18:37:35.788518 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.788365 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30ee2912-a7f7-4149-99e4-a53f2379003e" containerName="manager" Oct 30 18:37:35.788518 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.788424 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="30ee2912-a7f7-4149-99e4-a53f2379003e" containerName="manager" Oct 30 18:37:35.798235 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.798209 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:35.821605 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.821576 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-s47vd"] Oct 30 18:37:35.831979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.831953 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Oct 30 18:37:35.832124 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.832034 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6t6g4\"" Oct 30 18:37:35.842968 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.842943 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:37:35.846847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.846828 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:35.876490 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.876462 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:37:35.890821 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.890797 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-sgtpj\"" Oct 30 18:37:35.892095 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.892080 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Oct 30 18:37:35.898442 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.898406 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jzhdq"] Oct 30 18:37:35.901242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.901217 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc95w\" (UniqueName: \"kubernetes.io/projected/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-kube-api-access-gc95w\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:35.901341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.901253 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-cert\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:35.902130 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.902106 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:35.923999 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.923965 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Oct 30 18:37:35.923999 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.923981 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jmvdl\"" Oct 30 18:37:35.940616 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.940582 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jzhdq"] Oct 30 18:37:35.970422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.970383 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-ncjwb"] Oct 30 18:37:35.974081 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:35.974064 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.001829 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.001799 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.002026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.001854 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f7e082-b604-435a-b648-5f92ce5e4308-tls-certs\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.002026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.001985 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc95w\" (UniqueName: \"kubernetes.io/projected/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-kube-api-access-gc95w\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:36.002151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.002032 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smshj\" (UniqueName: \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.002151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.002066 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-cert\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:36.002260 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.002176 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmk2p\" (UniqueName: \"kubernetes.io/projected/48f7e082-b604-435a-b648-5f92ce5e4308-kube-api-access-rmk2p\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.005271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.005244 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-cert\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:36.017314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.017281 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-ncjwb"] Oct 30 18:37:36.024549 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.024501 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Oct 30 18:37:36.024715 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.024640 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Oct 30 18:37:36.024769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.024718 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-twbml\"" Oct 30 18:37:36.035115 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.033232 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc95w\" (UniqueName: \"kubernetes.io/projected/7a2f6848-9e7f-4ce4-be45-9fa160d4020e-kube-api-access-gc95w\") pod \"kserve-controller-manager-776cc66d47-s47vd\" (UID: \"7a2f6848-9e7f-4ce4-be45-9fa160d4020e\") " pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:36.102705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102619 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smshj\" (UniqueName: \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.102705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102664 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfxtb\" (UniqueName: \"kubernetes.io/projected/ad10e439-4eba-49aa-b98e-facac59da954-kube-api-access-zfxtb\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.102902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102709 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmk2p\" (UniqueName: \"kubernetes.io/projected/48f7e082-b604-435a-b648-5f92ce5e4308-kube-api-access-rmk2p\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.102902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102726 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-metrics-certs\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.102902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102753 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.102902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102797 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-cert\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.102902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.102865 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f7e082-b604-435a-b648-5f92ce5e4308-tls-certs\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.105238 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.105215 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f7e082-b604-435a-b648-5f92ce5e4308-tls-certs\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.105353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.105317 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.111159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.111137 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:36.127954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.127928 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smshj\" (UniqueName: \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\") pod \"llmisvc-controller-manager-869d5dcc5d-dp2zp\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.128647 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.128616 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmk2p\" (UniqueName: \"kubernetes.io/projected/48f7e082-b604-435a-b648-5f92ce5e4308-kube-api-access-rmk2p\") pod \"model-serving-api-75bd89fdfc-jzhdq\" (UID: \"48f7e082-b604-435a-b648-5f92ce5e4308\") " pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.160347 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.160309 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:36.204173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.204138 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-metrics-certs\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.204354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.204223 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-cert\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.204354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.204277 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfxtb\" (UniqueName: \"kubernetes.io/projected/ad10e439-4eba-49aa-b98e-facac59da954-kube-api-access-zfxtb\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.206919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.206890 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-cert\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.207039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.206931 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad10e439-4eba-49aa-b98e-facac59da954-metrics-certs\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.216503 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.216313 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:36.229990 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.229959 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfxtb\" (UniqueName: \"kubernetes.io/projected/ad10e439-4eba-49aa-b98e-facac59da954-kube-api-access-zfxtb\") pod \"odh-model-controller-769875db56-ncjwb\" (UID: \"ad10e439-4eba-49aa-b98e-facac59da954\") " pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.286633 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.286605 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:36.471386 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.471350 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-s47vd"] Oct 30 18:37:36.483772 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:36.483739 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2f6848_9e7f_4ce4_be45_9fa160d4020e.slice/crio-340b4a08b8c98ba8c460623738333b883eef637ca8a0b826ffa9d1201aa5fa2a WatchSource:0}: Error finding container 340b4a08b8c98ba8c460623738333b883eef637ca8a0b826ffa9d1201aa5fa2a: Status 404 returned error can't find the container with id 340b4a08b8c98ba8c460623738333b883eef637ca8a0b826ffa9d1201aa5fa2a Oct 30 18:37:36.523206 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.523177 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:37:36.523978 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:36.523954 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2abd3b5f_cf05_4ff8_8789_59d952a3830a.slice/crio-85507ebae9ef178a81eff74cc190ebbe970508e7ffebbf100c8549b409b8cdeb WatchSource:0}: Error finding container 85507ebae9ef178a81eff74cc190ebbe970508e7ffebbf100c8549b409b8cdeb: Status 404 returned error can't find the container with id 85507ebae9ef178a81eff74cc190ebbe970508e7ffebbf100c8549b409b8cdeb Oct 30 18:37:36.537519 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.537487 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jzhdq"] Oct 30 18:37:36.538821 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:36.538796 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f7e082_b604_435a_b648_5f92ce5e4308.slice/crio-31c14c5ec69fadfa3d1b7ab912c0c4dafad18137078486756ee17079b865ad1f WatchSource:0}: Error finding container 31c14c5ec69fadfa3d1b7ab912c0c4dafad18137078486756ee17079b865ad1f: Status 404 returned error can't find the container with id 31c14c5ec69fadfa3d1b7ab912c0c4dafad18137078486756ee17079b865ad1f Oct 30 18:37:36.600200 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:36.600176 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-ncjwb"] Oct 30 18:37:36.602142 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:36.602108 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad10e439_4eba_49aa_b98e_facac59da954.slice/crio-fe54986cb8b3d3b1742a5512a18bc7e692ad62695e3da97358729311be96e205 WatchSource:0}: Error finding container fe54986cb8b3d3b1742a5512a18bc7e692ad62695e3da97358729311be96e205: Status 404 returned error can't find the container with id fe54986cb8b3d3b1742a5512a18bc7e692ad62695e3da97358729311be96e205 Oct 30 18:37:37.338759 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:37.338509 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" event={"ID":"48f7e082-b604-435a-b648-5f92ce5e4308","Type":"ContainerStarted","Data":"31c14c5ec69fadfa3d1b7ab912c0c4dafad18137078486756ee17079b865ad1f"} Oct 30 18:37:37.341072 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:37.340986 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" event={"ID":"2abd3b5f-cf05-4ff8-8789-59d952a3830a","Type":"ContainerStarted","Data":"85507ebae9ef178a81eff74cc190ebbe970508e7ffebbf100c8549b409b8cdeb"} Oct 30 18:37:37.342961 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:37.342891 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" event={"ID":"7a2f6848-9e7f-4ce4-be45-9fa160d4020e","Type":"ContainerStarted","Data":"340b4a08b8c98ba8c460623738333b883eef637ca8a0b826ffa9d1201aa5fa2a"} Oct 30 18:37:37.345969 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:37.345938 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-ncjwb" event={"ID":"ad10e439-4eba-49aa-b98e-facac59da954","Type":"ContainerStarted","Data":"fe54986cb8b3d3b1742a5512a18bc7e692ad62695e3da97358729311be96e205"} Oct 30 18:37:39.363666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:39.363630 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" event={"ID":"48f7e082-b604-435a-b648-5f92ce5e4308","Type":"ContainerStarted","Data":"380aa1eea1c9052b01d351b9e7e74e7a3718cecb37cee7f438522e5ec26ab08f"} Oct 30 18:37:39.364491 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:39.364465 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:39.408659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:39.408608 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" podStartSLOduration=1.768900483 podStartE2EDuration="4.408589759s" totalImagesPullingTime="2.639689276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:39.407250541 +0000 UTC m=+330.851641559" watchObservedRunningTime="2026-10-30 18:37:39.408589759 +0000 UTC m=+330.852980757" Oct 30 18:37:40.370481 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.370443 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" event={"ID":"2abd3b5f-cf05-4ff8-8789-59d952a3830a","Type":"ContainerStarted","Data":"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be"} Oct 30 18:37:40.370918 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.370579 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:37:40.372165 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.372141 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" event={"ID":"7a2f6848-9e7f-4ce4-be45-9fa160d4020e","Type":"ContainerStarted","Data":"ac0f058771be4746d7f4044af22e403b1f5c55430c603ea5fa9dd7385afdd978"} Oct 30 18:37:40.372280 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.372193 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:37:40.373665 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.373643 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-ncjwb" event={"ID":"ad10e439-4eba-49aa-b98e-facac59da954","Type":"ContainerStarted","Data":"dc58fdaba08263bc428c028f1bb18cbf6bb42732c1fcd133753546e80e50127f"} Oct 30 18:37:40.373819 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.373803 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:40.478554 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.478491 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" podStartSLOduration=2.489031972 podStartE2EDuration="5.478474624s" totalImagesPullingTime="2.989442652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:40.478114653 +0000 UTC m=+331.922505674" watchObservedRunningTime="2026-10-30 18:37:40.478474624 +0000 UTC m=+331.922865621" Oct 30 18:37:40.479113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.479079 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" podStartSLOduration=2.668838296 podStartE2EDuration="5.479069005s" totalImagesPullingTime="2.810230709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:40.432640884 +0000 UTC m=+331.877031897" watchObservedRunningTime="2026-10-30 18:37:40.479069005 +0000 UTC m=+331.923460000" Oct 30 18:37:40.520278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:40.520211 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-ncjwb" podStartSLOduration=1.9936070479999999 podStartE2EDuration="5.520190542s" totalImagesPullingTime="3.526583494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:40.518874118 +0000 UTC m=+331.963265119" watchObservedRunningTime="2026-10-30 18:37:40.520190542 +0000 UTC m=+331.964581539" Oct 30 18:37:46.973137 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:37:46.973090 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:37:51.381471 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:51.381442 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-ncjwb" Oct 30 18:37:51.383583 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:51.383562 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-jzhdq" Oct 30 18:37:58.026227 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.026189 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-zsnt2"] Oct 30 18:37:58.029644 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.029625 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.050897 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.050861 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-zsnt2"] Oct 30 18:37:58.118661 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.118626 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st4q5\" (UniqueName: \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\") pod \"authorino-6f6944bcdd-zsnt2\" (UID: \"57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d\") " pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.219921 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.219871 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st4q5\" (UniqueName: \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\") pod \"authorino-6f6944bcdd-zsnt2\" (UID: \"57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d\") " pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.221123 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.221103 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-zsnt2"] Oct 30 18:37:58.221301 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:37:58.221283 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-st4q5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" podUID="57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d" Oct 30 18:37:58.249412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.249379 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st4q5\" (UniqueName: \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\") pod \"authorino-6f6944bcdd-zsnt2\" (UID: \"57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d\") " pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.252086 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.252062 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:37:58.256480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.256454 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.273474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.273428 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Oct 30 18:37:58.275449 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.275420 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:37:58.320904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.320815 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbd4h\" (UniqueName: \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.320904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.320876 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.422265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.422233 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbd4h\" (UniqueName: \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.422460 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.422294 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.425099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.425071 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.448007 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.447968 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbd4h\" (UniqueName: \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\") pod \"authorino-6889cbd8c4-9dh4s\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.468820 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.468794 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.470271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.470241 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:37:58.470478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.470463 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:37:58.482126 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.482104 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:58.508294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.508265 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:37:58.521133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.513233 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.544910 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.531840 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:37:58.623592 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.623568 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\" (UniqueName: \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\") pod \"57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d\" (UID: \"57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d\") " Oct 30 18:37:58.623826 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.623808 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7c4t\" (UniqueName: \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.623902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.623876 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.625399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.625377 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5" pod "57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d" (UID: "57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d"). InnerVolumeSpecName "kube-api-access-st4q5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:37:58.724738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.724706 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7c4t\" (UniqueName: \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.724923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.724757 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.724923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.724808 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st4q5\" (UniqueName: \"kubernetes.io/projected/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d-kube-api-access-st4q5\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:37:58.727132 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.727107 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.751770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.751743 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7c4t\" (UniqueName: \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\") pod \"authorino-f7f688f59-lvj75\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:58.783428 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.783399 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:37:58.785431 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:58.785405 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode17c883d_68ae_45a7_b1e2_5c292ab91689.slice/crio-539f861d815adecf3ff489ce95902983ba2ac5b925e64725c55ce1e9bb533bd1 WatchSource:0}: Error finding container 539f861d815adecf3ff489ce95902983ba2ac5b925e64725c55ce1e9bb533bd1: Status 404 returned error can't find the container with id 539f861d815adecf3ff489ce95902983ba2ac5b925e64725c55ce1e9bb533bd1 Oct 30 18:37:58.829749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:58.829719 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:37:59.132353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.132328 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:37:59.134499 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:37:59.134471 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eda35b3_e768_4439_8a7a_25c68b0c7da6.slice/crio-00f6efa41782e8f99e928dd33fd2d4f4ef27c54afd644fb597c6fa00fbedfdbf WatchSource:0}: Error finding container 00f6efa41782e8f99e928dd33fd2d4f4ef27c54afd644fb597c6fa00fbedfdbf: Status 404 returned error can't find the container with id 00f6efa41782e8f99e928dd33fd2d4f4ef27c54afd644fb597c6fa00fbedfdbf Oct 30 18:37:59.475529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.475496 6492 generic.go:356] "Generic (PLEG): container finished" podID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerID="ec1f0cf1ec8d7025f7e986d675c9d5e938df02b02e0b6a58eb9c5690f0e6298b" exitCode=0 Oct 30 18:37:59.475658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.475573 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" event={"ID":"ff54aab4-243f-4c0f-b941-5e2c6839ab77","Type":"ContainerDied","Data":"ec1f0cf1ec8d7025f7e986d675c9d5e938df02b02e0b6a58eb9c5690f0e6298b"} Oct 30 18:37:59.475995 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.475968 6492 scope.go:122] "RemoveContainer" containerID="ec1f0cf1ec8d7025f7e986d675c9d5e938df02b02e0b6a58eb9c5690f0e6298b" Oct 30 18:37:59.477068 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.477044 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" event={"ID":"e17c883d-68ae-45a7-b1e2-5c292ab91689","Type":"ContainerStarted","Data":"539f861d815adecf3ff489ce95902983ba2ac5b925e64725c55ce1e9bb533bd1"} Oct 30 18:37:59.477183 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.477111 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:37:59.477241 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.477187 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:37:59.477241 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.477221 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:37:59.478404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.478372 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7f688f59-lvj75" event={"ID":"2eda35b3-e768-4439-8a7a-25c68b0c7da6","Type":"ContainerStarted","Data":"00f6efa41782e8f99e928dd33fd2d4f4ef27c54afd644fb597c6fa00fbedfdbf"} Oct 30 18:37:59.478505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.478407 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-zsnt2" Oct 30 18:37:59.597256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.596787 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-zsnt2"] Oct 30 18:37:59.600453 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:37:59.600427 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-zsnt2"] Oct 30 18:38:00.485861 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.485810 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" event={"ID":"ff54aab4-243f-4c0f-b941-5e2c6839ab77","Type":"ContainerStarted","Data":"65faa4cbdf29ca2598cabdf14f45a44c930528689b5e6bd21250077450a0faca"} Oct 30 18:38:00.502561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.496872 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" podUID="e17c883d-68ae-45a7-b1e2-5c292ab91689" containerName="authorino" containerID="cri-o://a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964" gracePeriod=30 Oct 30 18:38:00.502561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.497174 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" event={"ID":"e17c883d-68ae-45a7-b1e2-5c292ab91689","Type":"ContainerStarted","Data":"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964"} Oct 30 18:38:00.534674 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.534635 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7f688f59-lvj75" event={"ID":"2eda35b3-e768-4439-8a7a-25c68b0c7da6","Type":"ContainerStarted","Data":"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b"} Oct 30 18:38:00.656472 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.655966 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-f7f688f59-lvj75" podStartSLOduration=2.180450168 podStartE2EDuration="2.655950924s" totalImagesPullingTime="475.500756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:00.653843358 +0000 UTC m=+352.098234373" watchObservedRunningTime="2026-10-30 18:38:00.655950924 +0000 UTC m=+352.100341899" Oct 30 18:38:00.663528 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.663497 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d" path="/var/lib/kubelet/pods/57dfd6ef-bf16-49b5-bdbe-d4ac7a2b773d/volumes" Oct 30 18:38:00.731425 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.731350 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" podStartSLOduration=2.055621167 podStartE2EDuration="2.731312322s" totalImagesPullingTime="675.691155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:00.726949162 +0000 UTC m=+352.171340159" watchObservedRunningTime="2026-10-30 18:38:00.731312322 +0000 UTC m=+352.175703330" Oct 30 18:38:00.732072 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.732044 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:38:00.732398 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.732377 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" podUID="9485a25c-0712-49a2-b47f-081392ab0cc7" containerName="authorino" containerID="cri-o://2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776" gracePeriod=30 Oct 30 18:38:00.857174 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.857149 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:38:00.949806 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.947397 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\" (UniqueName: \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\") pod \"e17c883d-68ae-45a7-b1e2-5c292ab91689\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " Oct 30 18:38:00.949806 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.947447 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\" (UniqueName: \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\") pod \"e17c883d-68ae-45a7-b1e2-5c292ab91689\" (UID: \"e17c883d-68ae-45a7-b1e2-5c292ab91689\") " Oct 30 18:38:00.952204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.952179 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h" pod "e17c883d-68ae-45a7-b1e2-5c292ab91689" (UID: "e17c883d-68ae-45a7-b1e2-5c292ab91689"). InnerVolumeSpecName "kube-api-access-nbd4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:00.960817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:00.960791 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert" pod "e17c883d-68ae-45a7-b1e2-5c292ab91689" (UID: "e17c883d-68ae-45a7-b1e2-5c292ab91689"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:01.040255 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.040231 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:38:01.048527 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.048498 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbd4h\" (UniqueName: \"kubernetes.io/projected/e17c883d-68ae-45a7-b1e2-5c292ab91689-kube-api-access-nbd4h\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:38:01.048640 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.048531 6492 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17c883d-68ae-45a7-b1e2-5c292ab91689-tls-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:38:01.149265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.149229 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\" (UniqueName: \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\") pod \"9485a25c-0712-49a2-b47f-081392ab0cc7\" (UID: \"9485a25c-0712-49a2-b47f-081392ab0cc7\") " Oct 30 18:38:01.151313 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.151283 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz" pod "9485a25c-0712-49a2-b47f-081392ab0cc7" (UID: "9485a25c-0712-49a2-b47f-081392ab0cc7"). InnerVolumeSpecName "kube-api-access-f8ffz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:01.250331 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.250296 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8ffz\" (UniqueName: \"kubernetes.io/projected/9485a25c-0712-49a2-b47f-081392ab0cc7-kube-api-access-f8ffz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:38:01.424824 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.424732 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:38:01.425974 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.425947 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Oct 30 18:38:01.426090 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.426009 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Oct 30 18:38:01.539927 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.539899 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:38:01.540549 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.540515 6492 generic.go:356] "Generic (PLEG): container finished" podID="e17c883d-68ae-45a7-b1e2-5c292ab91689" containerID="a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964" exitCode=0 Oct 30 18:38:01.540681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.540589 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" Oct 30 18:38:01.540681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.540589 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" event={"ID":"e17c883d-68ae-45a7-b1e2-5c292ab91689","Type":"ContainerDied","Data":"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964"} Oct 30 18:38:01.540681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.540635 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-9dh4s" event={"ID":"e17c883d-68ae-45a7-b1e2-5c292ab91689","Type":"ContainerDied","Data":"539f861d815adecf3ff489ce95902983ba2ac5b925e64725c55ce1e9bb533bd1"} Oct 30 18:38:01.540681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.540658 6492 scope.go:122] "RemoveContainer" containerID="a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964" Oct 30 18:38:01.542141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.542120 6492 generic.go:356] "Generic (PLEG): container finished" podID="9485a25c-0712-49a2-b47f-081392ab0cc7" containerID="2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776" exitCode=0 Oct 30 18:38:01.542233 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.542158 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" Oct 30 18:38:01.542233 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.542208 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" event={"ID":"9485a25c-0712-49a2-b47f-081392ab0cc7","Type":"ContainerDied","Data":"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776"} Oct 30 18:38:01.542316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.542246 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-jfhw9" event={"ID":"9485a25c-0712-49a2-b47f-081392ab0cc7","Type":"ContainerDied","Data":"fa74b2b186239e1f01b5d29529a3e549e9945b002d44f0c7177b77c142bb549e"} Oct 30 18:38:01.549754 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.549360 6492 scope.go:122] "RemoveContainer" containerID="a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964" Oct 30 18:38:01.549754 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:38:01.549702 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964\": container with ID starting with a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964 not found: ID does not exist" containerID="a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964" Oct 30 18:38:01.549754 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.549728 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964"} err="failed to get container status \"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964\": rpc error: code = NotFound desc = could not find container \"a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964\": container with ID starting with a5bb9c115aff3a3471e34eb7481221e6cc91ead09495fc7f05f68d796ca4f964 not found: ID does not exist" Oct 30 18:38:01.549754 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.549745 6492 scope.go:122] "RemoveContainer" containerID="2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776" Oct 30 18:38:01.557823 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.557807 6492 scope.go:122] "RemoveContainer" containerID="2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776" Oct 30 18:38:01.558077 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:38:01.558060 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776\": container with ID starting with 2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776 not found: ID does not exist" containerID="2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776" Oct 30 18:38:01.558123 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.558085 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776"} err="failed to get container status \"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776\": rpc error: code = NotFound desc = could not find container \"2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776\": container with ID starting with 2d477c178d7428148f988a7aa52f389626c94a1b35a9899ad89eed1673cf3776 not found: ID does not exist" Oct 30 18:38:01.591131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.591101 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:38:01.595466 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.595445 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-jfhw9"] Oct 30 18:38:01.633025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.632992 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:38:01.639155 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:01.639130 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-9dh4s"] Oct 30 18:38:02.428023 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:02.427974 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Oct 30 18:38:02.428233 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:02.428056 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Oct 30 18:38:02.663283 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:02.663250 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9485a25c-0712-49a2-b47f-081392ab0cc7" path="/var/lib/kubelet/pods/9485a25c-0712-49a2-b47f-081392ab0cc7/volumes" Oct 30 18:38:02.663675 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:02.663628 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e17c883d-68ae-45a7-b1e2-5c292ab91689" path="/var/lib/kubelet/pods/e17c883d-68ae-45a7-b1e2-5c292ab91689/volumes" Oct 30 18:38:03.424847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:03.424810 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Oct 30 18:38:03.425044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:03.424871 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Oct 30 18:38:04.424897 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:04.424850 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Oct 30 18:38:04.425279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:04.424917 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Oct 30 18:38:05.425398 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:05.425363 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Oct 30 18:38:05.425826 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:05.425435 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Oct 30 18:38:06.428990 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:06.428963 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:38:06.429848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:06.429830 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" Oct 30 18:38:11.381205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:11.381165 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:38:11.384133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:11.384108 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-s47vd" Oct 30 18:38:16.972031 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:38:16.971984 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:38:42.099864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.099822 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q"] Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100209 6492 cpu_manager.go:414] "containerMap: removing container" podUID="e17c883d-68ae-45a7-b1e2-5c292ab91689" containerName="authorino" Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100221 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e17c883d-68ae-45a7-b1e2-5c292ab91689" containerName="authorino" Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100238 6492 cpu_manager.go:414] "containerMap: removing container" podUID="9485a25c-0712-49a2-b47f-081392ab0cc7" containerName="authorino" Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100243 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9485a25c-0712-49a2-b47f-081392ab0cc7" containerName="authorino" Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100297 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="e17c883d-68ae-45a7-b1e2-5c292ab91689" containerName="authorino" Oct 30 18:38:42.100333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.100306 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="9485a25c-0712-49a2-b47f-081392ab0cc7" containerName="authorino" Oct 30 18:38:42.103620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.103605 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.119111 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.119088 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3140ad8-9fb8-45fe-9d54-fb94859c792f-cert\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.119250 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.119118 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qdq\" (UniqueName: \"kubernetes.io/projected/a3140ad8-9fb8-45fe-9d54-fb94859c792f-kube-api-access-d6qdq\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.142603 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.142573 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q"] Oct 30 18:38:42.220431 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.220377 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3140ad8-9fb8-45fe-9d54-fb94859c792f-cert\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.220431 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.220424 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qdq\" (UniqueName: \"kubernetes.io/projected/a3140ad8-9fb8-45fe-9d54-fb94859c792f-kube-api-access-d6qdq\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.222969 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.222938 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3140ad8-9fb8-45fe-9d54-fb94859c792f-cert\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.249286 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.249255 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qdq\" (UniqueName: \"kubernetes.io/projected/a3140ad8-9fb8-45fe-9d54-fb94859c792f-kube-api-access-d6qdq\") pod \"llmisvc-controller-manager-6466c554d8-pnb5q\" (UID: \"a3140ad8-9fb8-45fe-9d54-fb94859c792f\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.416511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.416398 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:42.734288 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.734251 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q"] Oct 30 18:38:42.745845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:42.745818 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" event={"ID":"a3140ad8-9fb8-45fe-9d54-fb94859c792f","Type":"ContainerStarted","Data":"cf8fac91f54d401a18769b279dcbda1aa4d031daf9f3721e5752e161816b5726"} Oct 30 18:38:43.752175 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:43.752139 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" event={"ID":"a3140ad8-9fb8-45fe-9d54-fb94859c792f","Type":"ContainerStarted","Data":"deb550651f985606f9ba683bec1792ead7265962e344ce30b9ee1df387f75a21"} Oct 30 18:38:43.752603 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:43.752205 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:38:43.798282 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:38:43.798228 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" podStartSLOduration=1.341120411 podStartE2EDuration="1.798213292s" totalImagesPullingTime="457.092881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:43.795237919 +0000 UTC m=+395.239628916" watchObservedRunningTime="2026-10-30 18:38:43.798213292 +0000 UTC m=+395.242604327" Oct 30 18:38:46.972938 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:38:46.972901 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:39:14.759189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:14.759155 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pnb5q" Oct 30 18:39:14.834863 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:14.834827 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:39:14.835068 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:14.835042 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" podUID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" containerName="manager" containerID="cri-o://f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be" gracePeriod=30 Oct 30 18:39:15.082135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.082111 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:39:15.215894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.215859 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\" (UniqueName: \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\") pod \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " Oct 30 18:39:15.216090 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.215913 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\" (UniqueName: \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\") pod \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\" (UID: \"2abd3b5f-cf05-4ff8-8789-59d952a3830a\") " Oct 30 18:39:15.218025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.217986 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj" pod "2abd3b5f-cf05-4ff8-8789-59d952a3830a" (UID: "2abd3b5f-cf05-4ff8-8789-59d952a3830a"). InnerVolumeSpecName "kube-api-access-smshj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:15.218025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.218004 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert" pod "2abd3b5f-cf05-4ff8-8789-59d952a3830a" (UID: "2abd3b5f-cf05-4ff8-8789-59d952a3830a"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:15.316684 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.316596 6492 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2abd3b5f-cf05-4ff8-8789-59d952a3830a-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:39:15.316684 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.316623 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smshj\" (UniqueName: \"kubernetes.io/projected/2abd3b5f-cf05-4ff8-8789-59d952a3830a-kube-api-access-smshj\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:39:15.910384 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.910348 6492 generic.go:356] "Generic (PLEG): container finished" podID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" containerID="f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be" exitCode=0 Oct 30 18:39:15.910869 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.910410 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" Oct 30 18:39:15.910869 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.910450 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" event={"ID":"2abd3b5f-cf05-4ff8-8789-59d952a3830a","Type":"ContainerDied","Data":"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be"} Oct 30 18:39:15.910869 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.910508 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp" event={"ID":"2abd3b5f-cf05-4ff8-8789-59d952a3830a","Type":"ContainerDied","Data":"85507ebae9ef178a81eff74cc190ebbe970508e7ffebbf100c8549b409b8cdeb"} Oct 30 18:39:15.910869 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.910531 6492 scope.go:122] "RemoveContainer" containerID="f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be" Oct 30 18:39:15.919999 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.919979 6492 scope.go:122] "RemoveContainer" containerID="f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be" Oct 30 18:39:15.920263 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:39:15.920242 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be\": container with ID starting with f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be not found: ID does not exist" containerID="f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be" Oct 30 18:39:15.920335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.920278 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be"} err="failed to get container status \"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be\": rpc error: code = NotFound desc = could not find container \"f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be\": container with ID starting with f7bd3fd2c5d5938a5f7f7b44df4e3c5e6c2d3ffc50c228d8e1dd0a344446c6be not found: ID does not exist" Oct 30 18:39:15.967382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.967347 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:39:15.978406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:15.978381 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dp2zp"] Oct 30 18:39:16.663531 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:39:16.663487 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" path="/var/lib/kubelet/pods/2abd3b5f-cf05-4ff8-8789-59d952a3830a/volumes" Oct 30 18:39:16.973006 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:39:16.972964 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:39:46.972557 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:39:46.972496 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:40:16.973906 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:40:16.973860 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:40:22.471757 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:22.471705 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:40:22.472309 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:22.471789 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:40:33.699914 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.699876 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5774ffdd7b-lhvxf"] Oct 30 18:40:33.700308 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.700245 6492 cpu_manager.go:414] "containerMap: removing container" podUID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" containerName="manager" Oct 30 18:40:33.700308 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.700257 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" containerName="manager" Oct 30 18:40:33.700383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.700339 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="2abd3b5f-cf05-4ff8-8789-59d952a3830a" containerName="manager" Oct 30 18:40:33.703412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.703389 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.723339 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.723309 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5774ffdd7b-lhvxf"] Oct 30 18:40:33.759529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.759499 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-tls-cert\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.759680 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.759535 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5nl\" (UniqueName: \"kubernetes.io/projected/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-kube-api-access-dh5nl\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.860383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.860346 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-tls-cert\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.860383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.860388 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5nl\" (UniqueName: \"kubernetes.io/projected/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-kube-api-access-dh5nl\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.862903 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.862882 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-tls-cert\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:33.884109 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:33.884077 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5nl\" (UniqueName: \"kubernetes.io/projected/2b098fa7-4c92-4a61-a0ac-f72f5f0659aa-kube-api-access-dh5nl\") pod \"authorino-5774ffdd7b-lhvxf\" (UID: \"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa\") " pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:34.015345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:34.015260 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" Oct 30 18:40:34.317185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:34.317085 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5774ffdd7b-lhvxf"] Oct 30 18:40:34.320116 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:40:34.320078 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b098fa7_4c92_4a61_a0ac_f72f5f0659aa.slice/crio-cfb7f8b8345a2036bde9672e21a39b16c88255b70c486490961947fa83ab7d77 WatchSource:0}: Error finding container cfb7f8b8345a2036bde9672e21a39b16c88255b70c486490961947fa83ab7d77: Status 404 returned error can't find the container with id cfb7f8b8345a2036bde9672e21a39b16c88255b70c486490961947fa83ab7d77 Oct 30 18:40:34.321402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:34.321382 6492 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:40:34.349511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:34.349463 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" event={"ID":"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa","Type":"ContainerStarted","Data":"cfb7f8b8345a2036bde9672e21a39b16c88255b70c486490961947fa83ab7d77"} Oct 30 18:40:35.356992 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.356950 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" event={"ID":"2b098fa7-4c92-4a61-a0ac-f72f5f0659aa","Type":"ContainerStarted","Data":"7f56abd30070fc3f3a3e307374cca6ae96895c9f91701f912dea1df4f465e69c"} Oct 30 18:40:35.399089 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.399034 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5774ffdd7b-lhvxf" podStartSLOduration=1.952413425 podStartE2EDuration="2.399021003s" totalImagesPullingTime="446.607578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:35.398608405 +0000 UTC m=+506.842999427" watchObservedRunningTime="2026-10-30 18:40:35.399021003 +0000 UTC m=+506.843411998" Oct 30 18:40:35.436182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.436140 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:40:35.436373 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.436334 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-f7f688f59-lvj75" podUID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" containerName="authorino" containerID="cri-o://a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b" gracePeriod=30 Oct 30 18:40:35.699865 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.699656 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:40:35.780967 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.780932 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\" (UniqueName: \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\") pod \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " Oct 30 18:40:35.781149 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.781038 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\" (UniqueName: \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\") pod \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\" (UID: \"2eda35b3-e768-4439-8a7a-25c68b0c7da6\") " Oct 30 18:40:35.783243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.783200 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t" pod "2eda35b3-e768-4439-8a7a-25c68b0c7da6" (UID: "2eda35b3-e768-4439-8a7a-25c68b0c7da6"). InnerVolumeSpecName "kube-api-access-j7c4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:40:35.792101 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.792046 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert" pod "2eda35b3-e768-4439-8a7a-25c68b0c7da6" (UID: "2eda35b3-e768-4439-8a7a-25c68b0c7da6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:40:35.881736 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.881708 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7c4t\" (UniqueName: \"kubernetes.io/projected/2eda35b3-e768-4439-8a7a-25c68b0c7da6-kube-api-access-j7c4t\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:40:35.881736 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:35.881734 6492 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2eda35b3-e768-4439-8a7a-25c68b0c7da6-tls-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:40:36.364175 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.364136 6492 generic.go:356] "Generic (PLEG): container finished" podID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" containerID="a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b" exitCode=0 Oct 30 18:40:36.364649 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.364195 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7f688f59-lvj75" Oct 30 18:40:36.364649 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.364225 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7f688f59-lvj75" event={"ID":"2eda35b3-e768-4439-8a7a-25c68b0c7da6","Type":"ContainerDied","Data":"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b"} Oct 30 18:40:36.364649 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.364269 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7f688f59-lvj75" event={"ID":"2eda35b3-e768-4439-8a7a-25c68b0c7da6","Type":"ContainerDied","Data":"00f6efa41782e8f99e928dd33fd2d4f4ef27c54afd644fb597c6fa00fbedfdbf"} Oct 30 18:40:36.364649 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.364286 6492 scope.go:122] "RemoveContainer" containerID="a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b" Oct 30 18:40:36.374124 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.374098 6492 scope.go:122] "RemoveContainer" containerID="a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b" Oct 30 18:40:36.374464 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:40:36.374434 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b\": container with ID starting with a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b not found: ID does not exist" containerID="a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b" Oct 30 18:40:36.374565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.374481 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b"} err="failed to get container status \"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b\": rpc error: code = NotFound desc = could not find container \"a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b\": container with ID starting with a003cf664ee9338a5a79e20b91b52d61416a922d972c911a027cdf435a20fc1b not found: ID does not exist" Oct 30 18:40:36.422886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.422849 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:40:36.426797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.426766 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f7f688f59-lvj75"] Oct 30 18:40:36.665662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:36.665572 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" path="/var/lib/kubelet/pods/2eda35b3-e768-4439-8a7a-25c68b0c7da6/volumes" Oct 30 18:40:46.972415 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:40:46.972375 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:40:53.139357 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.139262 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n"] Oct 30 18:40:53.139900 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.139881 6492 cpu_manager.go:414] "containerMap: removing container" podUID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" containerName="authorino" Oct 30 18:40:53.139940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.139902 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" containerName="authorino" Oct 30 18:40:53.139984 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.139975 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="2eda35b3-e768-4439-8a7a-25c68b0c7da6" containerName="authorino" Oct 30 18:40:53.143408 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.143391 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.172716 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.172681 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n"] Oct 30 18:40:53.246787 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.246746 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.246800 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.246877 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.246936 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593147ef-fd2e-4826-99a5-76094fcc971e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.246986 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ph27\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-kube-api-access-5ph27\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.247036 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247212 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.247106 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593147ef-fd2e-4826-99a5-76094fcc971e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247212 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.247147 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.247212 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.247197 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348271 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348312 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348524 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348432 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593147ef-fd2e-4826-99a5-76094fcc971e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348524 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348468 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ph27\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-kube-api-access-5ph27\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348534 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348610 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593147ef-fd2e-4826-99a5-76094fcc971e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348630 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348820 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348669 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348820 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348703 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348820 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348754 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348953 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.348987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.348971 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.349121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.349019 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.349329 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.349308 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/593147ef-fd2e-4826-99a5-76094fcc971e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.350929 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.350900 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/593147ef-fd2e-4826-99a5-76094fcc971e-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.351034 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.351014 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/593147ef-fd2e-4826-99a5-76094fcc971e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.372700 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.372671 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ph27\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-kube-api-access-5ph27\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.372818 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.372672 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/593147ef-fd2e-4826-99a5-76094fcc971e-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tvj2n\" (UID: \"593147ef-fd2e-4826-99a5-76094fcc971e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.456785 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.456748 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:53.778666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.778642 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n"] Oct 30 18:40:53.781145 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:40:53.781113 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593147ef_fd2e_4826_99a5_76094fcc971e.slice/crio-256269ac39f89d4a24a0a8ca32344e3d8a0ebcc5791ff942276cdac267c24176 WatchSource:0}: Error finding container 256269ac39f89d4a24a0a8ca32344e3d8a0ebcc5791ff942276cdac267c24176: Status 404 returned error can't find the container with id 256269ac39f89d4a24a0a8ca32344e3d8a0ebcc5791ff942276cdac267c24176 Oct 30 18:40:53.783382 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.783351 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:53.783429 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.783415 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:53.783462 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:53.783448 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:54.474699 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:54.474664 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" event={"ID":"593147ef-fd2e-4826-99a5-76094fcc971e","Type":"ContainerStarted","Data":"e7b43c90c048343ba9e89974b6304efecdb321b0e111e337f035222c97fda6b3"} Oct 30 18:40:54.475100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:54.474718 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" event={"ID":"593147ef-fd2e-4826-99a5-76094fcc971e","Type":"ContainerStarted","Data":"256269ac39f89d4a24a0a8ca32344e3d8a0ebcc5791ff942276cdac267c24176"} Oct 30 18:40:54.546385 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:54.546331 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podStartSLOduration=1.546315434 podStartE2EDuration="1.546315434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:54.532784824 +0000 UTC m=+525.977175842" watchObservedRunningTime="2026-10-30 18:40:54.546315434 +0000 UTC m=+525.990706431" Oct 30 18:40:55.457391 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:55.457346 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:40:55.458928 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:55.458899 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:40:55.459065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:55.458947 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:40:56.457559 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:56.457510 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:40:56.457975 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:56.457600 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:40:57.457872 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:57.457834 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:40:57.458288 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:57.457890 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:40:58.457536 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:58.457499 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:40:58.457755 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:58.457608 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:40:59.457872 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:59.457828 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:40:59.458277 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:40:59.457894 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:41:00.458081 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:00.458035 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tvj2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Oct 30 18:41:00.458485 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:00.458102 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" podUID="593147ef-fd2e-4826-99a5-76094fcc971e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Oct 30 18:41:01.831519 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:01.831485 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:41:01.832028 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:01.831952 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:41:01.832873 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:01.832850 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tvj2n" Oct 30 18:41:16.390941 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.390900 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:16.396485 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.396455 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.415165 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.415134 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:16.425388 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.425359 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ed302a08\"" Oct 30 18:41:16.458357 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.458320 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:16.462025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.461996 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.483240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.483208 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ed302a08-dockercfg-zzsk2\"" Oct 30 18:41:16.485181 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.485145 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:16.485730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.485704 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.485864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.485794 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cr4s\" (UniqueName: \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.586397 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.586353 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cr4s\" (UniqueName: \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.586397 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.586399 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.586712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.586459 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.586712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.586478 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf6f6\" (UniqueName: \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.587267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.587242 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.610117 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.610081 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cr4s\" (UniqueName: \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\") pod \"payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.687855 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.687750 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.687855 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.687835 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf6f6\" (UniqueName: \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.688530 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.688502 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.711302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.711264 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf6f6\" (UniqueName: \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\") pod \"payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.714117 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.714078 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:16.780205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:16.779146 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:16.975285 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:16.975243 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:41:17.025749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:17.025574 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:17.029680 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:17.029648 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf524c1_7106_457e_9df0_9cbeac83674a.slice/crio-af8e328bae3b75f7705078a6c2b4b6c8ccdf7fb4c3025bdbac2b4d5eabff4198 WatchSource:0}: Error finding container af8e328bae3b75f7705078a6c2b4b6c8ccdf7fb4c3025bdbac2b4d5eabff4198: Status 404 returned error can't find the container with id af8e328bae3b75f7705078a6c2b4b6c8ccdf7fb4c3025bdbac2b4d5eabff4198 Oct 30 18:41:17.119974 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:17.119945 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:17.122536 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:17.122491 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf6e869_4c30_4883_82cd_398f44e93a4e.slice/crio-beb17eed4a6e7f03664cc8441146eba1e70e800b2aa5aca69dba777f8c6b0f57 WatchSource:0}: Error finding container beb17eed4a6e7f03664cc8441146eba1e70e800b2aa5aca69dba777f8c6b0f57: Status 404 returned error can't find the container with id beb17eed4a6e7f03664cc8441146eba1e70e800b2aa5aca69dba777f8c6b0f57 Oct 30 18:41:17.636606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:17.636557 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" event={"ID":"cbf6e869-4c30-4883-82cd-398f44e93a4e","Type":"ContainerStarted","Data":"beb17eed4a6e7f03664cc8441146eba1e70e800b2aa5aca69dba777f8c6b0f57"} Oct 30 18:41:17.638314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:17.638279 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" event={"ID":"acf524c1-7106-457e-9df0-9cbeac83674a","Type":"ContainerStarted","Data":"af8e328bae3b75f7705078a6c2b4b6c8ccdf7fb4c3025bdbac2b4d5eabff4198"} Oct 30 18:41:19.653223 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.653185 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" event={"ID":"acf524c1-7106-457e-9df0-9cbeac83674a","Type":"ContainerStarted","Data":"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96"} Oct 30 18:41:19.653720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.653258 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:19.654886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.654863 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" event={"ID":"cbf6e869-4c30-4883-82cd-398f44e93a4e","Type":"ContainerStarted","Data":"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6"} Oct 30 18:41:19.654989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.654902 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:19.727552 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.727445 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" podStartSLOduration=1.284001125 podStartE2EDuration="3.727432386s" totalImagesPullingTime="2.443431261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:19.725852564 +0000 UTC m=+551.170243578" watchObservedRunningTime="2026-10-30 18:41:19.727432386 +0000 UTC m=+551.171823382" Oct 30 18:41:19.771943 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:19.771883 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" podStartSLOduration=1.365696756 podStartE2EDuration="3.771869319s" totalImagesPullingTime="2.406172563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:19.769635755 +0000 UTC m=+551.214026751" watchObservedRunningTime="2026-10-30 18:41:19.771869319 +0000 UTC m=+551.216260315" Oct 30 18:41:20.372202 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.372165 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:20.376121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.376099 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.398742 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.398700 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a33e5151\"" Oct 30 18:41:20.398885 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.398767 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a33e5151-dockercfg-2q4rp\"" Oct 30 18:41:20.399168 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.399153 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:20.429215 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.429178 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:20.437744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.433736 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.437744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.435271 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9lz\" (UniqueName: \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.437744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.435379 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.454599 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.454567 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:20.536469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.536432 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.536652 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.536476 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.536652 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.536566 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9lz\" (UniqueName: \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.536740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.536677 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwp8\" (UniqueName: \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.537456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.537440 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.563368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.563272 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9lz\" (UniqueName: \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\") pod \"payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.638169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.638059 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.638354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.638181 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwp8\" (UniqueName: \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.638989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.638960 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.662378 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.662347 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwp8\" (UniqueName: \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\") pod \"payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:20.689271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.689238 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:20.750706 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:20.750670 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:21.002182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.002115 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:21.004573 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:21.004523 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dad270_a9ff_4690_8276_09a90217fa06.slice/crio-148b1b58257e13aad92da739e2f2e147fcf0d5abd448f8dba684d75c82d2d3a4 WatchSource:0}: Error finding container 148b1b58257e13aad92da739e2f2e147fcf0d5abd448f8dba684d75c82d2d3a4: Status 404 returned error can't find the container with id 148b1b58257e13aad92da739e2f2e147fcf0d5abd448f8dba684d75c82d2d3a4 Oct 30 18:41:21.073606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.073581 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:21.076174 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:21.076149 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc60985c_918f_4fec_a75b_ae298d065e40.slice/crio-f0a2ffc7cad21b45c8c9e0c25c94006dc772b97a77e299ef4a219fe490a248c9 WatchSource:0}: Error finding container f0a2ffc7cad21b45c8c9e0c25c94006dc772b97a77e299ef4a219fe490a248c9: Status 404 returned error can't find the container with id f0a2ffc7cad21b45c8c9e0c25c94006dc772b97a77e299ef4a219fe490a248c9 Oct 30 18:41:21.666655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.666612 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" event={"ID":"cc60985c-918f-4fec-a75b-ae298d065e40","Type":"ContainerStarted","Data":"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696"} Oct 30 18:41:21.667119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.666680 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" event={"ID":"cc60985c-918f-4fec-a75b-ae298d065e40","Type":"ContainerStarted","Data":"f0a2ffc7cad21b45c8c9e0c25c94006dc772b97a77e299ef4a219fe490a248c9"} Oct 30 18:41:21.667119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.666725 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:21.668521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.668487 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" event={"ID":"d9dad270-a9ff-4690-8276-09a90217fa06","Type":"ContainerStarted","Data":"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58"} Oct 30 18:41:21.668709 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.668529 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" event={"ID":"d9dad270-a9ff-4690-8276-09a90217fa06","Type":"ContainerStarted","Data":"148b1b58257e13aad92da739e2f2e147fcf0d5abd448f8dba684d75c82d2d3a4"} Oct 30 18:41:21.668709 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.668580 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:21.709662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.709609 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" podStartSLOduration=1.709595609 podStartE2EDuration="1.709595609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:21.708020111 +0000 UTC m=+553.152411107" watchObservedRunningTime="2026-10-30 18:41:21.709595609 +0000 UTC m=+553.153986605" Oct 30 18:41:21.752764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:21.752710 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" podStartSLOduration=1.75269639 podStartE2EDuration="1.75269639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:21.751838402 +0000 UTC m=+553.196229398" watchObservedRunningTime="2026-10-30 18:41:21.75269639 +0000 UTC m=+553.197087385" Oct 30 18:41:30.017069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.017029 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp"] Oct 30 18:41:30.020587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.020565 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.048364 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.048331 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:41:30.048573 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.048367 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:41:30.048573 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.048489 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:41:30.048717 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.048623 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:41:30.051475 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.051447 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp"] Oct 30 18:41:30.135887 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.135832 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk8zx\" (UniqueName: \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.136100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.135896 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.136100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.136006 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.237264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.237216 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.237465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.237370 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk8zx\" (UniqueName: \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.237465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.237403 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.238077 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.238051 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.239726 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.239706 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.261446 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.261413 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk8zx\" (UniqueName: \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\") pod \"maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.335301 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.335204 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:30.641776 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.641638 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp"] Oct 30 18:41:30.644918 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:30.644892 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda148010b_18f9_4aa4_bb3b_38639b4112fa.slice/crio-aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88 WatchSource:0}: Error finding container aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88: Status 404 returned error can't find the container with id aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88 Oct 30 18:41:30.665228 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.665201 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:30.665397 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.665368 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:30.719446 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:30.719395 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" event={"ID":"a148010b-18f9-4aa4-bb3b-38639b4112fa","Type":"ContainerStarted","Data":"aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88"} Oct 30 18:41:32.677182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:32.677147 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:32.677618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:32.677231 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:33.741981 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:33.741943 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" event={"ID":"a148010b-18f9-4aa4-bb3b-38639b4112fa","Type":"ContainerStarted","Data":"fa522dfca07c5c04b7feb3031c3cfe3cd1dcbbd99d334ede59aac761827de39e"} Oct 30 18:41:33.783462 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:33.783409 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" podStartSLOduration=2.406343229 podStartE2EDuration="4.783395358s" totalImagesPullingTime="2.377052129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:33.782635102 +0000 UTC m=+565.227026114" watchObservedRunningTime="2026-10-30 18:41:33.783395358 +0000 UTC m=+565.227786392" Oct 30 18:41:34.748704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:34.748670 6492 generic.go:356] "Generic (PLEG): container finished" podID="a148010b-18f9-4aa4-bb3b-38639b4112fa" containerID="fa522dfca07c5c04b7feb3031c3cfe3cd1dcbbd99d334ede59aac761827de39e" exitCode=0 Oct 30 18:41:34.749113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:34.748742 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" event={"ID":"a148010b-18f9-4aa4-bb3b-38639b4112fa","Type":"ContainerDied","Data":"fa522dfca07c5c04b7feb3031c3cfe3cd1dcbbd99d334ede59aac761827de39e"} Oct 30 18:41:35.875960 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.875918 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5"] Oct 30 18:41:35.879826 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.879804 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.892910 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.892884 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:35.894133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.894081 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.894248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.894209 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4t4p\" (UniqueName: \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.894297 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.894265 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.906966 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.906936 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5"] Oct 30 18:41:35.994947 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.994905 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\" (UniqueName: \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\") pod \"a148010b-18f9-4aa4-bb3b-38639b4112fa\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " Oct 30 18:41:35.995151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995047 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\") pod \"a148010b-18f9-4aa4-bb3b-38639b4112fa\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " Oct 30 18:41:35.995151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995117 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\") pod \"a148010b-18f9-4aa4-bb3b-38639b4112fa\" (UID: \"a148010b-18f9-4aa4-bb3b-38639b4112fa\") " Oct 30 18:41:35.995293 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995274 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.995357 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995332 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t4p\" (UniqueName: \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.995415 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995390 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.995473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.995446 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca" pod "a148010b-18f9-4aa4-bb3b-38639b4112fa" (UID: "a148010b-18f9-4aa4-bb3b-38639b4112fa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:35.996240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.996211 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:35.997595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.997556 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx" pod "a148010b-18f9-4aa4-bb3b-38639b4112fa" (UID: "a148010b-18f9-4aa4-bb3b-38639b4112fa"). InnerVolumeSpecName "kube-api-access-wk8zx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:35.997595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.997572 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca" pod "a148010b-18f9-4aa4-bb3b-38639b4112fa" (UID: "a148010b-18f9-4aa4-bb3b-38639b4112fa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:35.997758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:35.997632 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:36.019846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.019811 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4t4p\" (UniqueName: \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\") pod \"maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:36.096989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.096933 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk8zx\" (UniqueName: \"kubernetes.io/projected/a148010b-18f9-4aa4-bb3b-38639b4112fa-kube-api-access-wk8zx\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:36.096989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.096979 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:36.096989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.096994 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a148010b-18f9-4aa4-bb3b-38639b4112fa-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:36.208692 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.208654 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:36.516351 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.516321 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5"] Oct 30 18:41:36.518737 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:36.518703 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31115e0_4927_4a9f_bf4f_56f47ea95fe9.slice/crio-5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f WatchSource:0}: Error finding container 5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f: Status 404 returned error can't find the container with id 5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f Oct 30 18:41:36.761353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.761322 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" Oct 30 18:41:36.761565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.761320 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp" event={"ID":"a148010b-18f9-4aa4-bb3b-38639b4112fa","Type":"ContainerDied","Data":"aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88"} Oct 30 18:41:36.761565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.761462 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aafc4742caf52c9b2d1eab40a4ad37712de3dbb4edd8a064404e59fd0db74e88" Oct 30 18:41:36.762827 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:36.762801 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" event={"ID":"b31115e0-4927-4a9f-bf4f-56f47ea95fe9","Type":"ContainerStarted","Data":"5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f"} Oct 30 18:41:37.771346 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.771311 6492 generic.go:356] "Generic (PLEG): container finished" podID="b31115e0-4927-4a9f-bf4f-56f47ea95fe9" containerID="08f9767608721878f98a679761191265d07a8b9bc7331ab0e2ae6b0aa11d81a2" exitCode=0 Oct 30 18:41:37.771769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.771394 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" event={"ID":"b31115e0-4927-4a9f-bf4f-56f47ea95fe9","Type":"ContainerDied","Data":"08f9767608721878f98a679761191265d07a8b9bc7331ab0e2ae6b0aa11d81a2"} Oct 30 18:41:37.994195 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.994154 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5"] Oct 30 18:41:37.994606 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.994592 6492 cpu_manager.go:414] "containerMap: removing container" podUID="a148010b-18f9-4aa4-bb3b-38639b4112fa" containerName="revoke-keys" Oct 30 18:41:37.994658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.994609 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a148010b-18f9-4aa4-bb3b-38639b4112fa" containerName="revoke-keys" Oct 30 18:41:37.994702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.994693 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="a148010b-18f9-4aa4-bb3b-38639b4112fa" containerName="revoke-keys" Oct 30 18:41:37.997886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:37.997870 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.015765 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.015734 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjlvx\" (UniqueName: \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.015927 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.015793 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.015991 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.015916 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.019273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.019243 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5"] Oct 30 18:41:38.117412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.117289 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjlvx\" (UniqueName: \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.117412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.117374 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.117704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.117439 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.118204 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.118180 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.119842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.119817 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.141189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.141160 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjlvx\" (UniqueName: \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\") pod \"maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.310938 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.310898 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:38.643890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.643865 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5"] Oct 30 18:41:38.646972 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:38.646924 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de920d1_cf98_4cbb_9e38_66ab9c41451b.slice/crio-27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6 WatchSource:0}: Error finding container 27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6: Status 404 returned error can't find the container with id 27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6 Oct 30 18:41:38.778294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.778257 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" event={"ID":"3de920d1-cf98-4cbb-9e38-66ab9c41451b","Type":"ContainerStarted","Data":"27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6"} Oct 30 18:41:38.981900 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:38.981874 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:39.036878 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.036846 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\") pod \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " Oct 30 18:41:39.037075 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.036903 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\") pod \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " Oct 30 18:41:39.037075 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.036940 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\" (UniqueName: \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\") pod \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\" (UID: \"b31115e0-4927-4a9f-bf4f-56f47ea95fe9\") " Oct 30 18:41:39.037322 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.037293 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca" pod "b31115e0-4927-4a9f-bf4f-56f47ea95fe9" (UID: "b31115e0-4927-4a9f-bf4f-56f47ea95fe9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:39.039116 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.039085 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca" pod "b31115e0-4927-4a9f-bf4f-56f47ea95fe9" (UID: "b31115e0-4927-4a9f-bf4f-56f47ea95fe9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:39.039239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.039159 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p" pod "b31115e0-4927-4a9f-bf4f-56f47ea95fe9" (UID: "b31115e0-4927-4a9f-bf4f-56f47ea95fe9"). InnerVolumeSpecName "kube-api-access-r4t4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:39.138528 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.138481 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:39.138528 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.138524 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4t4p\" (UniqueName: \"kubernetes.io/projected/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-kube-api-access-r4t4p\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:39.138528 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.138534 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31115e0-4927-4a9f-bf4f-56f47ea95fe9-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:39.785102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.785008 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" Oct 30 18:41:39.785102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.785048 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5" event={"ID":"b31115e0-4927-4a9f-bf4f-56f47ea95fe9","Type":"ContainerDied","Data":"5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f"} Oct 30 18:41:39.785102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.785097 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5abbafe0abc86ddd654c245746bef12d09d259d686b9be5d1b369da346b7d47f" Oct 30 18:41:39.786795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.786770 6492 generic.go:356] "Generic (PLEG): container finished" podID="3de920d1-cf98-4cbb-9e38-66ab9c41451b" containerID="2f42edaadd08f93d89eccf3ed385050074b88a2d3295ab41abfc38ad5c60dc9b" exitCode=0 Oct 30 18:41:39.786934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:39.786799 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" event={"ID":"3de920d1-cf98-4cbb-9e38-66ab9c41451b","Type":"ContainerDied","Data":"2f42edaadd08f93d89eccf3ed385050074b88a2d3295ab41abfc38ad5c60dc9b"} Oct 30 18:41:40.944097 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:40.944067 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:41.054672 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.054565 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\") pod \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " Oct 30 18:41:41.054672 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.054620 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\" (UniqueName: \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\") pod \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " Oct 30 18:41:41.054672 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.054664 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\") pod \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\" (UID: \"3de920d1-cf98-4cbb-9e38-66ab9c41451b\") " Oct 30 18:41:41.055051 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.055021 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca" pod "3de920d1-cf98-4cbb-9e38-66ab9c41451b" (UID: "3de920d1-cf98-4cbb-9e38-66ab9c41451b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:41.056871 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.056841 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca" pod "3de920d1-cf98-4cbb-9e38-66ab9c41451b" (UID: "3de920d1-cf98-4cbb-9e38-66ab9c41451b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:41.057018 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.056881 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx" pod "3de920d1-cf98-4cbb-9e38-66ab9c41451b" (UID: "3de920d1-cf98-4cbb-9e38-66ab9c41451b"). InnerVolumeSpecName "kube-api-access-mjlvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:41.156210 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.156169 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:41.156210 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.156202 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjlvx\" (UniqueName: \"kubernetes.io/projected/3de920d1-cf98-4cbb-9e38-66ab9c41451b-kube-api-access-mjlvx\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:41.156210 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.156213 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3de920d1-cf98-4cbb-9e38-66ab9c41451b-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:41.798712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.798681 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" Oct 30 18:41:41.798892 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.798681 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5" event={"ID":"3de920d1-cf98-4cbb-9e38-66ab9c41451b","Type":"ContainerDied","Data":"27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6"} Oct 30 18:41:41.798892 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:41.798801 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27fe6d75e90dd4794dfdc4a4af83db04421d796975bdd8e3c74df0e2804b96d6" Oct 30 18:41:42.908443 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908407 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt"] Oct 30 18:41:42.908867 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908852 6492 cpu_manager.go:414] "containerMap: removing container" podUID="3de920d1-cf98-4cbb-9e38-66ab9c41451b" containerName="revoke-keys" Oct 30 18:41:42.908867 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908864 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3de920d1-cf98-4cbb-9e38-66ab9c41451b" containerName="revoke-keys" Oct 30 18:41:42.908939 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908873 6492 cpu_manager.go:414] "containerMap: removing container" podUID="b31115e0-4927-4a9f-bf4f-56f47ea95fe9" containerName="revoke-keys" Oct 30 18:41:42.908939 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908878 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b31115e0-4927-4a9f-bf4f-56f47ea95fe9" containerName="revoke-keys" Oct 30 18:41:42.908998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908939 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="3de920d1-cf98-4cbb-9e38-66ab9c41451b" containerName="revoke-keys" Oct 30 18:41:42.908998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.908945 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="b31115e0-4927-4a9f-bf4f-56f47ea95fe9" containerName="revoke-keys" Oct 30 18:41:42.912770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.912747 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:42.942271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.942191 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:41:42.942434 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.942354 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:41:42.942434 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.942358 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:41:42.942555 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.942442 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:41:42.944590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.944566 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt"] Oct 30 18:41:42.974276 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.974238 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2tv5\" (UniqueName: \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:42.974469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.974282 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:42.974469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:42.974361 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.075680 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.075641 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2tv5\" (UniqueName: \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.075847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.075687 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.075847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.075812 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.076457 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.076437 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.078017 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.077997 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.099352 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.099322 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2tv5\" (UniqueName: \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\") pod \"maas-api-revoke-keys-e2e-derive-a33e5151-82hjt\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.225637 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.225600 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:43.528583 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.528556 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt"] Oct 30 18:41:43.530322 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:43.530294 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e54297_095d_46b7_a6da_57e5925058aa.slice/crio-b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac WatchSource:0}: Error finding container b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac: Status 404 returned error can't find the container with id b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac Oct 30 18:41:43.809716 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:43.809683 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" event={"ID":"20e54297-095d-46b7-a6da-57e5925058aa","Type":"ContainerStarted","Data":"b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac"} Oct 30 18:41:44.816926 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:44.816890 6492 generic.go:356] "Generic (PLEG): container finished" podID="20e54297-095d-46b7-a6da-57e5925058aa" containerID="85c2eb70f55389312f7d88fd82481bbe0b20121a4d878123784f96cd2bb17650" exitCode=0 Oct 30 18:41:44.817336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:44.816959 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" event={"ID":"20e54297-095d-46b7-a6da-57e5925058aa","Type":"ContainerDied","Data":"85c2eb70f55389312f7d88fd82481bbe0b20121a4d878123784f96cd2bb17650"} Oct 30 18:41:45.979161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:45.979128 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:46.106480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.106382 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\" (UniqueName: \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\") pod \"20e54297-095d-46b7-a6da-57e5925058aa\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " Oct 30 18:41:46.106480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.106427 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\") pod \"20e54297-095d-46b7-a6da-57e5925058aa\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " Oct 30 18:41:46.106480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.106471 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\") pod \"20e54297-095d-46b7-a6da-57e5925058aa\" (UID: \"20e54297-095d-46b7-a6da-57e5925058aa\") " Oct 30 18:41:46.106995 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.106964 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca" pod "20e54297-095d-46b7-a6da-57e5925058aa" (UID: "20e54297-095d-46b7-a6da-57e5925058aa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:46.108656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.108629 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca" pod "20e54297-095d-46b7-a6da-57e5925058aa" (UID: "20e54297-095d-46b7-a6da-57e5925058aa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:46.108783 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.108769 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5" pod "20e54297-095d-46b7-a6da-57e5925058aa" (UID: "20e54297-095d-46b7-a6da-57e5925058aa"). InnerVolumeSpecName "kube-api-access-n2tv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:46.208093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.208046 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2tv5\" (UniqueName: \"kubernetes.io/projected/20e54297-095d-46b7-a6da-57e5925058aa-kube-api-access-n2tv5\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.208093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.208090 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.208355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.208106 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20e54297-095d-46b7-a6da-57e5925058aa-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.830319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.830291 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" Oct 30 18:41:46.830319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.830299 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt" event={"ID":"20e54297-095d-46b7-a6da-57e5925058aa","Type":"ContainerDied","Data":"b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac"} Oct 30 18:41:46.830534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:46.830348 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8627d9c44599b3c539ed62c0047f461e50e3b1a36da22cfc0143e6d8992faac" Oct 30 18:41:46.973107 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:46.973067 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:41:48.449732 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.449696 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:48.497115 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.497080 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:48.532142 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.532104 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp"] Oct 30 18:41:48.534986 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.534953 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b77c9bba-7hnnp"] Oct 30 18:41:48.667163 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.667114 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a148010b-18f9-4aa4-bb3b-38639b4112fa" path="/var/lib/kubelet/pods/a148010b-18f9-4aa4-bb3b-38639b4112fa/volumes" Oct 30 18:41:48.938824 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.938735 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5"] Oct 30 18:41:48.942049 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:48.942015 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-48afdd9e-99mb5"] Oct 30 18:41:50.078222 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:50.078179 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5"] Oct 30 18:41:50.090839 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:50.090802 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed302a08-7kbp5"] Oct 30 18:41:50.665292 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:50.665254 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3de920d1-cf98-4cbb-9e38-66ab9c41451b" path="/var/lib/kubelet/pods/3de920d1-cf98-4cbb-9e38-66ab9c41451b/volumes" Oct 30 18:41:50.665867 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:50.665844 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b31115e0-4927-4a9f-bf4f-56f47ea95fe9" path="/var/lib/kubelet/pods/b31115e0-4927-4a9f-bf4f-56f47ea95fe9/volumes" Oct 30 18:41:53.450123 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.450081 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" podUID="acf524c1-7106-457e-9df0-9cbeac83674a" containerName="payload-processing" containerID="cri-o://cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96" gracePeriod=25 Oct 30 18:41:53.498207 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.498152 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" podUID="cbf6e869-4c30-4883-82cd-398f44e93a4e" containerName="payload-pre-processing" containerID="cri-o://f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6" gracePeriod=25 Oct 30 18:41:53.532854 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.532822 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:53.569100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.569056 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:53.805831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.805803 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:53.809433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.809406 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:53.874594 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.874518 6492 generic.go:356] "Generic (PLEG): container finished" podID="acf524c1-7106-457e-9df0-9cbeac83674a" containerID="cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96" exitCode=0 Oct 30 18:41:53.874594 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.874584 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" Oct 30 18:41:53.874594 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.874587 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" event={"ID":"acf524c1-7106-457e-9df0-9cbeac83674a","Type":"ContainerDied","Data":"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96"} Oct 30 18:41:53.874924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.874642 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt" event={"ID":"acf524c1-7106-457e-9df0-9cbeac83674a","Type":"ContainerDied","Data":"af8e328bae3b75f7705078a6c2b4b6c8ccdf7fb4c3025bdbac2b4d5eabff4198"} Oct 30 18:41:53.874924 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.874665 6492 scope.go:122] "RemoveContainer" containerID="cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96" Oct 30 18:41:53.875997 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.875967 6492 generic.go:356] "Generic (PLEG): container finished" podID="cbf6e869-4c30-4883-82cd-398f44e93a4e" containerID="f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6" exitCode=0 Oct 30 18:41:53.876128 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.876022 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" Oct 30 18:41:53.876128 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.876035 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" event={"ID":"cbf6e869-4c30-4883-82cd-398f44e93a4e","Type":"ContainerDied","Data":"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6"} Oct 30 18:41:53.876128 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.876072 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p" event={"ID":"cbf6e869-4c30-4883-82cd-398f44e93a4e","Type":"ContainerDied","Data":"beb17eed4a6e7f03664cc8441146eba1e70e800b2aa5aca69dba777f8c6b0f57"} Oct 30 18:41:53.883925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.883907 6492 scope.go:122] "RemoveContainer" containerID="cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96" Oct 30 18:41:53.884240 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:53.884212 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96\": container with ID starting with cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96 not found: ID does not exist" containerID="cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96" Oct 30 18:41:53.884325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.884249 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96"} err="failed to get container status \"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96\": rpc error: code = NotFound desc = could not find container \"cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96\": container with ID starting with cc4d7d17e9905f5f3be9a5346192eab39e4d476ece9bcf0e99c5214e493cfe96 not found: ID does not exist" Oct 30 18:41:53.884325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.884273 6492 scope.go:122] "RemoveContainer" containerID="f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6" Oct 30 18:41:53.891580 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.891554 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\" (UniqueName: \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\") pod \"acf524c1-7106-457e-9df0-9cbeac83674a\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " Oct 30 18:41:53.891743 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.891623 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\" (UniqueName: \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\") pod \"cbf6e869-4c30-4883-82cd-398f44e93a4e\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " Oct 30 18:41:53.891743 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.891658 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\") pod \"acf524c1-7106-457e-9df0-9cbeac83674a\" (UID: \"acf524c1-7106-457e-9df0-9cbeac83674a\") " Oct 30 18:41:53.891743 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.891732 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\") pod \"cbf6e869-4c30-4883-82cd-398f44e93a4e\" (UID: \"cbf6e869-4c30-4883-82cd-398f44e93a4e\") " Oct 30 18:41:53.892106 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.892086 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume" pod "acf524c1-7106-457e-9df0-9cbeac83674a" (UID: "acf524c1-7106-457e-9df0-9cbeac83674a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:53.892247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.892217 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume" pod "cbf6e869-4c30-4883-82cd-398f44e93a4e" (UID: "cbf6e869-4c30-4883-82cd-398f44e93a4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:53.893870 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.893847 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s" pod "acf524c1-7106-457e-9df0-9cbeac83674a" (UID: "acf524c1-7106-457e-9df0-9cbeac83674a"). InnerVolumeSpecName "kube-api-access-9cr4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:53.893966 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.893924 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6" pod "cbf6e869-4c30-4883-82cd-398f44e93a4e" (UID: "cbf6e869-4c30-4883-82cd-398f44e93a4e"). InnerVolumeSpecName "kube-api-access-cf6f6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:53.894194 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.894179 6492 scope.go:122] "RemoveContainer" containerID="f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6" Oct 30 18:41:53.894518 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:53.894493 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6\": container with ID starting with f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6 not found: ID does not exist" containerID="f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6" Oct 30 18:41:53.894597 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.894525 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6"} err="failed to get container status \"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6\": rpc error: code = NotFound desc = could not find container \"f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6\": container with ID starting with f44838cf6c90f29dfc35362ba531bf6bb6b580f84ccf13184cd3ae7c0194d4c6 not found: ID does not exist" Oct 30 18:41:53.993323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.993238 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbf6e869-4c30-4883-82cd-398f44e93a4e-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.993323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.993268 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cr4s\" (UniqueName: \"kubernetes.io/projected/acf524c1-7106-457e-9df0-9cbeac83674a-kube-api-access-9cr4s\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.993323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.993277 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf6f6\" (UniqueName: \"kubernetes.io/projected/cbf6e869-4c30-4883-82cd-398f44e93a4e-kube-api-access-cf6f6\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.993323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:53.993286 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acf524c1-7106-457e-9df0-9cbeac83674a-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.226057 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.226014 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:54.234393 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.234364 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ed302a08-847f99dd5b-79tbt"] Oct 30 18:41:54.269915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.269826 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:54.283295 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.283255 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed302a08-7bd8df4f9-m9k6p"] Oct 30 18:41:54.664441 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.664360 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acf524c1-7106-457e-9df0-9cbeac83674a" path="/var/lib/kubelet/pods/acf524c1-7106-457e-9df0-9cbeac83674a/volumes" Oct 30 18:41:54.664825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:54.664756 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbf6e869-4c30-4883-82cd-398f44e93a4e" path="/var/lib/kubelet/pods/cbf6e869-4c30-4883-82cd-398f44e93a4e/volumes" Oct 30 18:41:55.082712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.082674 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:41:55.083138 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083119 6492 cpu_manager.go:414] "containerMap: removing container" podUID="acf524c1-7106-457e-9df0-9cbeac83674a" containerName="payload-processing" Oct 30 18:41:55.083249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083139 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acf524c1-7106-457e-9df0-9cbeac83674a" containerName="payload-processing" Oct 30 18:41:55.083249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083161 6492 cpu_manager.go:414] "containerMap: removing container" podUID="cbf6e869-4c30-4883-82cd-398f44e93a4e" containerName="payload-pre-processing" Oct 30 18:41:55.083249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083167 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbf6e869-4c30-4883-82cd-398f44e93a4e" containerName="payload-pre-processing" Oct 30 18:41:55.083249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083198 6492 cpu_manager.go:414] "containerMap: removing container" podUID="20e54297-095d-46b7-a6da-57e5925058aa" containerName="revoke-keys" Oct 30 18:41:55.083249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083203 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20e54297-095d-46b7-a6da-57e5925058aa" containerName="revoke-keys" Oct 30 18:41:55.083418 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083275 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbf6e869-4c30-4883-82cd-398f44e93a4e" containerName="payload-pre-processing" Oct 30 18:41:55.083418 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083290 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="20e54297-095d-46b7-a6da-57e5925058aa" containerName="revoke-keys" Oct 30 18:41:55.083418 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.083300 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="acf524c1-7106-457e-9df0-9cbeac83674a" containerName="payload-processing" Oct 30 18:41:55.088190 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.088168 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.107958 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.107920 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:41:55.117923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.117866 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-a47385\"" Oct 30 18:41:55.117923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.117891 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-a47385-dockercfg-95s4v\"" Oct 30 18:41:55.162265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.162226 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:41:55.166171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.166141 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.189280 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.189248 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:41:55.205637 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.205595 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gdmg\" (UniqueName: \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.205824 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.205647 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.306480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.306443 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gdmg\" (UniqueName: \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.306480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.306487 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.306766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.306644 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.306766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.306746 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwl5b\" (UniqueName: \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.308267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.308237 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.330453 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.330423 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gdmg\" (UniqueName: \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\") pod \"payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.402209 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.402110 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.407920 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.407883 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.408070 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.407958 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwl5b\" (UniqueName: \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.408687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.408662 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.431610 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.431578 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwl5b\" (UniqueName: \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\") pod \"payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.484903 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.484686 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.713011 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.712980 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:41:55.715174 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:55.715145 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ac06a6_6b24_404c_bc49_393723f87c1b.slice/crio-a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e WatchSource:0}: Error finding container a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e: Status 404 returned error can't find the container with id a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e Oct 30 18:41:55.805775 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.805748 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:41:55.807323 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:55.807294 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93ad3c0_ae40_40ad_bfe1_479dca147d8f.slice/crio-d45310b8623277ef583d280a46e4b36a46060c33e2b2fd5139be4aa33fa03334 WatchSource:0}: Error finding container d45310b8623277ef583d280a46e4b36a46060c33e2b2fd5139be4aa33fa03334: Status 404 returned error can't find the container with id d45310b8623277ef583d280a46e4b36a46060c33e2b2fd5139be4aa33fa03334 Oct 30 18:41:55.901192 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.901156 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" event={"ID":"f93ad3c0-ae40-40ad-bfe1-479dca147d8f","Type":"ContainerStarted","Data":"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8"} Oct 30 18:41:55.901354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.901210 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" event={"ID":"f93ad3c0-ae40-40ad-bfe1-479dca147d8f","Type":"ContainerStarted","Data":"d45310b8623277ef583d280a46e4b36a46060c33e2b2fd5139be4aa33fa03334"} Oct 30 18:41:55.901354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.901263 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:41:55.902949 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.902927 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" event={"ID":"11ac06a6-6b24-404c-bc49-393723f87c1b","Type":"ContainerStarted","Data":"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310"} Oct 30 18:41:55.903030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.902965 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" event={"ID":"11ac06a6-6b24-404c-bc49-393723f87c1b","Type":"ContainerStarted","Data":"a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e"} Oct 30 18:41:55.903030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.903018 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:41:55.951913 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.951862 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" podStartSLOduration=0.951846602 podStartE2EDuration="951.846602ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:55.9507079 +0000 UTC m=+587.395098903" watchObservedRunningTime="2026-10-30 18:41:55.951846602 +0000 UTC m=+587.396237579" Oct 30 18:41:55.995048 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:55.994926 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" podStartSLOduration=0.994912217 podStartE2EDuration="994.912217ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:55.994477917 +0000 UTC m=+587.438868912" watchObservedRunningTime="2026-10-30 18:41:55.994912217 +0000 UTC m=+587.439303212" Oct 30 18:41:56.023171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:56.023136 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt"] Oct 30 18:41:56.037121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:56.037090 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a33e5151-82hjt"] Oct 30 18:41:56.664744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:56.664703 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20e54297-095d-46b7-a6da-57e5925058aa" path="/var/lib/kubelet/pods/20e54297-095d-46b7-a6da-57e5925058aa/volumes" Oct 30 18:41:57.393739 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.393704 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:41:57.397451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.397430 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.429315 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.429281 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-0ddc26\"" Oct 30 18:41:57.435529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.435491 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:41:57.527856 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.527815 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.528033 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.527914 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q499d\" (UniqueName: \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.629018 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.628974 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q499d\" (UniqueName: \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.629167 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.629097 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.630055 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.630027 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.666335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.666239 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q499d\" (UniqueName: \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\") pod \"payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:57.713637 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:57.713595 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:58.027979 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.027942 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:41:58.029202 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:41:58.029173 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3243bc3_ea41_48cf_9304_ee914c9aec07.slice/crio-8b37f3aba5b2fd5da5823e2ae3c0c156a25aedf979843082049d333655970f4a WatchSource:0}: Error finding container 8b37f3aba5b2fd5da5823e2ae3c0c156a25aedf979843082049d333655970f4a: Status 404 returned error can't find the container with id 8b37f3aba5b2fd5da5823e2ae3c0c156a25aedf979843082049d333655970f4a Oct 30 18:41:58.533278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.533224 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" podUID="d9dad270-a9ff-4690-8276-09a90217fa06" containerName="payload-processing" containerID="cri-o://b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58" gracePeriod=25 Oct 30 18:41:58.569515 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.569458 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" podUID="cc60985c-918f-4fec-a75b-ae298d065e40" containerName="payload-pre-processing" containerID="cri-o://5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696" gracePeriod=25 Oct 30 18:41:58.848625 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.848596 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:58.874249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.874212 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:58.925958 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.925913 6492 generic.go:356] "Generic (PLEG): container finished" podID="d9dad270-a9ff-4690-8276-09a90217fa06" containerID="b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58" exitCode=0 Oct 30 18:41:58.926160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.925980 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" Oct 30 18:41:58.926216 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.926188 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" event={"ID":"d9dad270-a9ff-4690-8276-09a90217fa06","Type":"ContainerDied","Data":"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58"} Oct 30 18:41:58.926273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.926256 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr" event={"ID":"d9dad270-a9ff-4690-8276-09a90217fa06","Type":"ContainerDied","Data":"148b1b58257e13aad92da739e2f2e147fcf0d5abd448f8dba684d75c82d2d3a4"} Oct 30 18:41:58.926326 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.926291 6492 scope.go:122] "RemoveContainer" containerID="b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58" Oct 30 18:41:58.929314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.929288 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" event={"ID":"b3243bc3-ea41-48cf-9304-ee914c9aec07","Type":"ContainerStarted","Data":"6bd46fccaa1c7b10284e71c2d794e06d105f5c1f918e633eceae0ed8782d1d34"} Oct 30 18:41:58.929432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.929342 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:41:58.929432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.929366 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" event={"ID":"b3243bc3-ea41-48cf-9304-ee914c9aec07","Type":"ContainerStarted","Data":"8b37f3aba5b2fd5da5823e2ae3c0c156a25aedf979843082049d333655970f4a"} Oct 30 18:41:58.930487 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.930461 6492 generic.go:356] "Generic (PLEG): container finished" podID="cc60985c-918f-4fec-a75b-ae298d065e40" containerID="5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696" exitCode=0 Oct 30 18:41:58.930620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.930563 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" event={"ID":"cc60985c-918f-4fec-a75b-ae298d065e40","Type":"ContainerDied","Data":"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696"} Oct 30 18:41:58.930620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.930607 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" event={"ID":"cc60985c-918f-4fec-a75b-ae298d065e40","Type":"ContainerDied","Data":"f0a2ffc7cad21b45c8c9e0c25c94006dc772b97a77e299ef4a219fe490a248c9"} Oct 30 18:41:58.930723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.930675 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p" Oct 30 18:41:58.943575 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.941449 6492 scope.go:122] "RemoveContainer" containerID="b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58" Oct 30 18:41:58.943716 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:58.943664 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58\": container with ID starting with b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58 not found: ID does not exist" containerID="b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58" Oct 30 18:41:58.943716 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.943701 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58"} err="failed to get container status \"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58\": rpc error: code = NotFound desc = could not find container \"b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58\": container with ID starting with b534f07b1fd72071b72bbcc481967e361a23393cf8542199a031ad666c228b58 not found: ID does not exist" Oct 30 18:41:58.943854 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.943725 6492 scope.go:122] "RemoveContainer" containerID="5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696" Oct 30 18:41:58.944333 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944308 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\") pod \"cc60985c-918f-4fec-a75b-ae298d065e40\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " Oct 30 18:41:58.944607 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944581 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\") pod \"d9dad270-a9ff-4690-8276-09a90217fa06\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " Oct 30 18:41:58.944770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944746 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume" pod "cc60985c-918f-4fec-a75b-ae298d065e40" (UID: "cc60985c-918f-4fec-a75b-ae298d065e40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:58.944770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944765 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\" (UniqueName: \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\") pod \"cc60985c-918f-4fec-a75b-ae298d065e40\" (UID: \"cc60985c-918f-4fec-a75b-ae298d065e40\") " Oct 30 18:41:58.944915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944801 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\" (UniqueName: \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\") pod \"d9dad270-a9ff-4690-8276-09a90217fa06\" (UID: \"d9dad270-a9ff-4690-8276-09a90217fa06\") " Oct 30 18:41:58.945006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.944981 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume" pod "d9dad270-a9ff-4690-8276-09a90217fa06" (UID: "d9dad270-a9ff-4690-8276-09a90217fa06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:58.945099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.945078 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc60985c-918f-4fec-a75b-ae298d065e40-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:58.945327 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.945307 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9dad270-a9ff-4690-8276-09a90217fa06-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:58.961702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.961668 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8" pod "cc60985c-918f-4fec-a75b-ae298d065e40" (UID: "cc60985c-918f-4fec-a75b-ae298d065e40"). InnerVolumeSpecName "kube-api-access-fwwp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:58.961989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.961711 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz" pod "d9dad270-a9ff-4690-8276-09a90217fa06" (UID: "d9dad270-a9ff-4690-8276-09a90217fa06"). InnerVolumeSpecName "kube-api-access-9r9lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:58.964045 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.964025 6492 scope.go:122] "RemoveContainer" containerID="5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696" Oct 30 18:41:58.964344 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:41:58.964313 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696\": container with ID starting with 5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696 not found: ID does not exist" containerID="5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696" Oct 30 18:41:58.964444 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:58.964350 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696"} err="failed to get container status \"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696\": rpc error: code = NotFound desc = could not find container \"5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696\": container with ID starting with 5cb2502e03d512d6814d4cc801efe94e04718c5cac6d43fab028589ea7630696 not found: ID does not exist" Oct 30 18:41:59.014335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.014283 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" podStartSLOduration=2.014268616 podStartE2EDuration="2.014268616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:59.013815687 +0000 UTC m=+590.458206685" watchObservedRunningTime="2026-10-30 18:41:59.014268616 +0000 UTC m=+590.458659612" Oct 30 18:41:59.047024 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.046987 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwwp8\" (UniqueName: \"kubernetes.io/projected/cc60985c-918f-4fec-a75b-ae298d065e40-kube-api-access-fwwp8\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:59.047024 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.047018 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r9lz\" (UniqueName: \"kubernetes.io/projected/d9dad270-a9ff-4690-8276-09a90217fa06-kube-api-access-9r9lz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:41:59.281427 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.281386 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:59.285400 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.285366 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a33e5151-85fc5ccf7b-nzgfr"] Oct 30 18:41:59.319463 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.319404 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:59.323080 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.323047 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a33e5151-6655fc458c-lkp5p"] Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.945895 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.946725 6492 cpu_manager.go:414] "containerMap: removing container" podUID="d9dad270-a9ff-4690-8276-09a90217fa06" containerName="payload-processing" Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.946743 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9dad270-a9ff-4690-8276-09a90217fa06" containerName="payload-processing" Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.946793 6492 cpu_manager.go:414] "containerMap: removing container" podUID="cc60985c-918f-4fec-a75b-ae298d065e40" containerName="payload-pre-processing" Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.946801 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc60985c-918f-4fec-a75b-ae298d065e40" containerName="payload-pre-processing" Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.946980 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9dad270-a9ff-4690-8276-09a90217fa06" containerName="payload-processing" Oct 30 18:41:59.948842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.947003 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc60985c-918f-4fec-a75b-ae298d065e40" containerName="payload-pre-processing" Oct 30 18:41:59.952449 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.952427 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:41:59.968530 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.968503 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:41:59.971283 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:41:59.971260 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-b2ef42\"" Oct 30 18:42:00.004433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.004398 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:42:00.007980 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.007964 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.024025 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.024002 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-b2ef42-dockercfg-xxxpq\"" Oct 30 18:42:00.029475 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.029447 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:42:00.057416 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.057381 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfjhf\" (UniqueName: \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.057578 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.057435 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.159088 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.159042 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc46b\" (UniqueName: \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.159305 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.159097 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.159305 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.159275 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfjhf\" (UniqueName: \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.159399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.159319 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.160263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.160243 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.182690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.182651 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfjhf\" (UniqueName: \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\") pod \"payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.259987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.259890 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc46b\" (UniqueName: \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.259987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.259940 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.260676 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.260656 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.266715 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.266695 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.283620 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.283595 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc46b\" (UniqueName: \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\") pod \"payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.322005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.321445 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.571808 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.571780 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:42:00.573675 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:00.573648 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c92a0a_fa51_49a1_b6a7_738d74001a9f.slice/crio-03b240db0ce46603c06c339d6042d6e161b7832f9dba40bf4d5548cde7854b2d WatchSource:0}: Error finding container 03b240db0ce46603c06c339d6042d6e161b7832f9dba40bf4d5548cde7854b2d: Status 404 returned error can't find the container with id 03b240db0ce46603c06c339d6042d6e161b7832f9dba40bf4d5548cde7854b2d Oct 30 18:42:00.644552 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.644517 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:42:00.646027 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:00.646002 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa26e794_20f3_46cc_985e_6e97ed4c5599.slice/crio-d32d308837cd668fc59ad1856be92344a83d25f531f204eae26b991610783713 WatchSource:0}: Error finding container d32d308837cd668fc59ad1856be92344a83d25f531f204eae26b991610783713: Status 404 returned error can't find the container with id d32d308837cd668fc59ad1856be92344a83d25f531f204eae26b991610783713 Oct 30 18:42:00.667998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.667901 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc60985c-918f-4fec-a75b-ae298d065e40" path="/var/lib/kubelet/pods/cc60985c-918f-4fec-a75b-ae298d065e40/volumes" Oct 30 18:42:00.668432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.668406 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9dad270-a9ff-4690-8276-09a90217fa06" path="/var/lib/kubelet/pods/d9dad270-a9ff-4690-8276-09a90217fa06/volumes" Oct 30 18:42:00.923966 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.923881 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:42:00.927153 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.927131 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:00.950214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.949417 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" event={"ID":"fa26e794-20f3-46cc-985e-6e97ed4c5599","Type":"ContainerStarted","Data":"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd"} Oct 30 18:42:00.950214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.949476 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" event={"ID":"fa26e794-20f3-46cc-985e-6e97ed4c5599","Type":"ContainerStarted","Data":"d32d308837cd668fc59ad1856be92344a83d25f531f204eae26b991610783713"} Oct 30 18:42:00.950214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.949587 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:00.950865 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.950844 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-f6f7d5-dockercfg-gs2tr\"" Oct 30 18:42:00.951031 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.950845 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-f6f7d5\"" Oct 30 18:42:00.952516 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.952494 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" event={"ID":"51c92a0a-fa51-49a1-b6a7-738d74001a9f","Type":"ContainerStarted","Data":"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18"} Oct 30 18:42:00.952638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.952534 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" event={"ID":"51c92a0a-fa51-49a1-b6a7-738d74001a9f","Type":"ContainerStarted","Data":"03b240db0ce46603c06c339d6042d6e161b7832f9dba40bf4d5548cde7854b2d"} Oct 30 18:42:00.952638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.952608 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:00.953689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.953667 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:42:00.998359 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:00.998325 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:42:01.002558 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.002520 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.024783 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.024732 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" podStartSLOduration=2.024719914 podStartE2EDuration="2.024719914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:01.023732524 +0000 UTC m=+592.468123522" watchObservedRunningTime="2026-10-30 18:42:01.024719914 +0000 UTC m=+592.469110907" Oct 30 18:42:01.038028 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.037991 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:42:01.066319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.066275 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbs2z\" (UniqueName: \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.066498 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.066410 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.108469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.108409 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" podStartSLOduration=2.108397573 podStartE2EDuration="2.108397573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:01.106434191 +0000 UTC m=+592.550825213" watchObservedRunningTime="2026-10-30 18:42:01.108397573 +0000 UTC m=+592.552788569" Oct 30 18:42:01.167961 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.167929 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbs2z\" (UniqueName: \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.168150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.167982 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.168150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.168006 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.168150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.168074 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qv8w\" (UniqueName: \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.168695 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.168670 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.192222 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.192152 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbs2z\" (UniqueName: \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\") pod \"payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.241023 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.240986 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:01.269775 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.269741 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qv8w\" (UniqueName: \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.269943 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.269828 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.270562 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.270517 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.294267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.294236 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qv8w\" (UniqueName: \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\") pod \"payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.315929 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.315892 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.563551 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.563510 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:42:01.565869 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:01.565844 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d5008_fd97_47ad_81cf_64385b25936d.slice/crio-edd65c667e10053fa34346ba842b4a39c52e1c32b17bafeb5644854fcf4b6452 WatchSource:0}: Error finding container edd65c667e10053fa34346ba842b4a39c52e1c32b17bafeb5644854fcf4b6452: Status 404 returned error can't find the container with id edd65c667e10053fa34346ba842b4a39c52e1c32b17bafeb5644854fcf4b6452 Oct 30 18:42:01.639888 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.639858 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:42:01.640593 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:01.640568 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9a251c_c576_4e6c_87fa_6d295f0c7833.slice/crio-537c9a257b85c4375d7835a2a88be43d2a78d6c292ddf7d8878d948203031969 WatchSource:0}: Error finding container 537c9a257b85c4375d7835a2a88be43d2a78d6c292ddf7d8878d948203031969: Status 404 returned error can't find the container with id 537c9a257b85c4375d7835a2a88be43d2a78d6c292ddf7d8878d948203031969 Oct 30 18:42:01.961470 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.961419 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" event={"ID":"ed9a251c-c576-4e6c-87fa-6d295f0c7833","Type":"ContainerStarted","Data":"60ddc01f64011be3ea03069688a74b6aa2afb88ed36f12bfca8ef4424e695dde"} Oct 30 18:42:01.961965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.961493 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" event={"ID":"ed9a251c-c576-4e6c-87fa-6d295f0c7833","Type":"ContainerStarted","Data":"537c9a257b85c4375d7835a2a88be43d2a78d6c292ddf7d8878d948203031969"} Oct 30 18:42:01.962228 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.962193 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:01.974337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.974252 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" event={"ID":"434d5008-fd97-47ad-81cf-64385b25936d","Type":"ContainerStarted","Data":"3288d643743855344ab00cf392c52e30e6d06197538dd590248e8bc48f9688be"} Oct 30 18:42:01.974337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.974319 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" event={"ID":"434d5008-fd97-47ad-81cf-64385b25936d","Type":"ContainerStarted","Data":"edd65c667e10053fa34346ba842b4a39c52e1c32b17bafeb5644854fcf4b6452"} Oct 30 18:42:01.974781 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:01.974759 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:02.025005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:02.024939 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" podStartSLOduration=2.024921134 podStartE2EDuration="2.024921134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:02.023828918 +0000 UTC m=+593.468219934" watchObservedRunningTime="2026-10-30 18:42:02.024921134 +0000 UTC m=+593.469312133" Oct 30 18:42:02.069907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:02.069856 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" podStartSLOduration=2.069841983 podStartE2EDuration="2.069841983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:02.068673643 +0000 UTC m=+593.513064639" watchObservedRunningTime="2026-10-30 18:42:02.069841983 +0000 UTC m=+593.514233010" Oct 30 18:42:04.611918 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.611881 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:42:04.615657 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.615636 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.639217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.639186 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:42:04.645689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.645657 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-a759a1\"" Oct 30 18:42:04.645857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.645743 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-a759a1-dockercfg-fwcsl\"" Oct 30 18:42:04.707876 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.707839 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.708061 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.707989 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfxbm\" (UniqueName: \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.808774 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.808724 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.808957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.808849 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfxbm\" (UniqueName: \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.809674 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.809649 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.835992 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.835955 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfxbm\" (UniqueName: \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\") pod \"payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:04.931079 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:04.930981 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:05.236973 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:05.236940 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:42:05.240488 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:05.240444 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97bb660_8c09_4628_b654_144b0bf021cd.slice/crio-1b6cc717aa361f1b6f45aabe08d981361f7263f3b2f53b117d7acb89d43573d5 WatchSource:0}: Error finding container 1b6cc717aa361f1b6f45aabe08d981361f7263f3b2f53b117d7acb89d43573d5: Status 404 returned error can't find the container with id 1b6cc717aa361f1b6f45aabe08d981361f7263f3b2f53b117d7acb89d43573d5 Oct 30 18:42:06.002630 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.002588 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" event={"ID":"a97bb660-8c09-4628-b654-144b0bf021cd","Type":"ContainerStarted","Data":"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0"} Oct 30 18:42:06.003058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.002651 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" event={"ID":"a97bb660-8c09-4628-b654-144b0bf021cd","Type":"ContainerStarted","Data":"1b6cc717aa361f1b6f45aabe08d981361f7263f3b2f53b117d7acb89d43573d5"} Oct 30 18:42:06.003058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.002696 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:06.046026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.045972 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" podStartSLOduration=2.045956668 podStartE2EDuration="2.045956668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:06.04490986 +0000 UTC m=+597.489300875" watchObservedRunningTime="2026-10-30 18:42:06.045956668 +0000 UTC m=+597.490347664" Oct 30 18:42:06.912277 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.912244 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:42:06.912451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:06.912406 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:42:08.757838 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:08.757811 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:42:08.759653 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:08.759623 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:42:09.945561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:09.945518 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:42:11.977564 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:11.977499 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:42:11.978005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:11.977928 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:42:12.984676 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:12.984648 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:42:12.985154 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:12.984719 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:42:16.978572 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:42:16.978515 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:42:17.016591 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:17.016561 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:42:28.594030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.593995 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:42:28.598360 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.598335 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.619292 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.619255 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"kube-root-ca.crt\"" Oct 30 18:42:28.629491 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.629462 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"openshift-service-ca.crt\"" Oct 30 18:42:28.629702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.629686 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"default-dockercfg-szghr\"" Oct 30 18:42:28.629783 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.629719 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"simulator-w2-b2ef42-kserve-self-signed-certs\"" Oct 30 18:42:28.631571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.631535 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:42:28.650306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650275 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.650446 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650317 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.650446 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650349 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhvjk\" (UniqueName: \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.650569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650464 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.650569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650487 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.650891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.650628 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.751901 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.751853 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.751918 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.751969 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvjk\" (UniqueName: \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752073 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752562 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752520 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752661 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752475 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752661 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752598 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752746 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752667 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.752902 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.752880 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.754888 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.754853 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.755389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.755367 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.775917 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.775888 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhvjk\" (UniqueName: \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\") pod \"simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:28.912126 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:28.912028 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:29.219184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:29.219158 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:42:29.221607 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:29.221575 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4091c9d8_60d3_4b63_a9f7_673a3cf89bf3.slice/crio-af562f70121314ecb1eae322d4260ef49650383cb356af1dcede1b54a7788f5b WatchSource:0}: Error finding container af562f70121314ecb1eae322d4260ef49650383cb356af1dcede1b54a7788f5b: Status 404 returned error can't find the container with id af562f70121314ecb1eae322d4260ef49650383cb356af1dcede1b54a7788f5b Oct 30 18:42:30.166294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:30.166236 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" event={"ID":"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3","Type":"ContainerStarted","Data":"af562f70121314ecb1eae322d4260ef49650383cb356af1dcede1b54a7788f5b"} Oct 30 18:42:31.174051 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:31.174013 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" event={"ID":"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3","Type":"ContainerStarted","Data":"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760"} Oct 30 18:42:31.220173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:31.220100 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" podStartSLOduration=2.109082696 podStartE2EDuration="3.220079248s" totalImagesPullingTime="1.110996552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:31.219518982 +0000 UTC m=+622.663909979" watchObservedRunningTime="2026-10-30 18:42:31.220079248 +0000 UTC m=+622.664470244" Oct 30 18:42:31.912417 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:31.912380 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:34.045920 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.045886 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:42:34.051027 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.050992 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.079158 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.079120 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"kube-root-ca.crt\"" Oct 30 18:42:34.079314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.079158 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"openshift-service-ca.crt\"" Oct 30 18:42:34.079314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.079171 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"simulator-w0-f6f7d5-kserve-self-signed-certs\"" Oct 30 18:42:34.079314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.079224 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"default-dockercfg-5q6fx\"" Oct 30 18:42:34.082595 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.082568 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:42:34.211051 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211005 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.211257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211075 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.211257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211210 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.211381 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211266 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.211381 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211319 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.211564 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.211402 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j565b\" (UniqueName: \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312186 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312061 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312186 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312130 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312186 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312154 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312254 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312346 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312425 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j565b\" (UniqueName: \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312684 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312569 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312684 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312589 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.312684 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.312662 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.314657 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.314625 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.314814 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.314799 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.336231 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.336202 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j565b\" (UniqueName: \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\") pod \"simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.364210 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.364163 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:34.672155 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:34.672125 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:42:34.673677 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:34.673649 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf957d7c_0d1d_4b6c_92a1_be785a4c16a0.slice/crio-bd1e1a20fd5b48a4e17c1944558f00b989a5a6bba8321518f9c093f9be56d958 WatchSource:0}: Error finding container bd1e1a20fd5b48a4e17c1944558f00b989a5a6bba8321518f9c093f9be56d958: Status 404 returned error can't find the container with id bd1e1a20fd5b48a4e17c1944558f00b989a5a6bba8321518f9c093f9be56d958 Oct 30 18:42:35.203099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:35.203061 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" event={"ID":"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0","Type":"ContainerStarted","Data":"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1"} Oct 30 18:42:35.203681 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:35.203112 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" event={"ID":"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0","Type":"ContainerStarted","Data":"bd1e1a20fd5b48a4e17c1944558f00b989a5a6bba8321518f9c093f9be56d958"} Oct 30 18:42:35.246240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:35.246169 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" podStartSLOduration=1.246148336 podStartE2EDuration="1.246148336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:35.244302955 +0000 UTC m=+626.688693976" watchObservedRunningTime="2026-10-30 18:42:35.246148336 +0000 UTC m=+626.690539330" Oct 30 18:42:35.365155 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:35.365116 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:38.912329 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:38.912290 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:38.926686 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:38.926657 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:39.239398 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:39.239371 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:42:44.364829 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:44.364786 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:44.378048 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:44.378021 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:45.089591 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.089526 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:42:45.105003 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.104967 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.120440 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.120407 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:42:45.130666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.130627 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"premium-w4-a47385-kserve-self-signed-certs\"" Oct 30 18:42:45.130851 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.130704 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"default-dockercfg-658qm\"" Oct 30 18:42:45.133220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.133194 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"kube-root-ca.crt\"" Oct 30 18:42:45.133220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.133194 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"openshift-service-ca.crt\"" Oct 30 18:42:45.226039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.225989 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.226039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.226035 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlqk6\" (UniqueName: \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.226296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.226109 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.226296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.226154 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.226296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.226274 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.226452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.226330 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.281779 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.281748 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.326851 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.326901 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlqk6\" (UniqueName: \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.326958 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327003 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327099 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327134 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327703 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327356 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.327894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327868 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.328032 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.327961 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.329748 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.329722 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.330325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.330297 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.350968 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.350905 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlqk6\" (UniqueName: \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\") pod \"premium-w4-a47385-kserve-c6db4bdbb-j4g95\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.420386 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.420348 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:45.724460 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:45.724429 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:42:45.726410 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:42:45.726368 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfae2421f_38e9_4124_a446_75085b4b1da9.slice/crio-aaa81838571dc0805fb68772469fecad7965c7f246a83100a1f3e2e364bfd5de WatchSource:0}: Error finding container aaa81838571dc0805fb68772469fecad7965c7f246a83100a1f3e2e364bfd5de: Status 404 returned error can't find the container with id aaa81838571dc0805fb68772469fecad7965c7f246a83100a1f3e2e364bfd5de Oct 30 18:42:46.277716 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:46.277664 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" event={"ID":"fae2421f-38e9-4124-a446-75085b4b1da9","Type":"ContainerStarted","Data":"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f"} Oct 30 18:42:46.277904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:46.277720 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" event={"ID":"fae2421f-38e9-4124-a446-75085b4b1da9","Type":"ContainerStarted","Data":"aaa81838571dc0805fb68772469fecad7965c7f246a83100a1f3e2e364bfd5de"} Oct 30 18:42:46.320501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:46.320441 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" podStartSLOduration=1.3204261480000001 podStartE2EDuration="1.320426148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:46.319137228 +0000 UTC m=+637.763528225" watchObservedRunningTime="2026-10-30 18:42:46.320426148 +0000 UTC m=+637.764817144" Oct 30 18:42:46.421394 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:46.421352 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:46.974770 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:42:46.974730 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:42:55.420805 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:55.420760 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:55.433767 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:55.433740 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:42:56.360687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:42:56.360648 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:43:16.976620 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:43:16.976577 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:43:46.975004 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:43:46.974963 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:43:47.157218 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.157184 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts"] Oct 30 18:43:47.161269 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.161252 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.177391 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.177362 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:43:47.189860 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.189822 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:43:47.190026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.189936 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:43:47.190026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.190014 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:43:47.192324 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.192298 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts"] Oct 30 18:43:47.296662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.296576 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdzqz\" (UniqueName: \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.296662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.296619 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.296850 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.296804 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.397569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.397501 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.397770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.397597 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdzqz\" (UniqueName: \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.397985 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.397953 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.398384 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.398347 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.400898 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.400872 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.421762 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.421728 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdzqz\" (UniqueName: \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\") pod \"maas-api-revoke-sub-59c67ac2-5m4ts\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.475140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.475112 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:47.786416 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:47.786388 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts"] Oct 30 18:43:47.788283 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:43:47.788250 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd219385_41d8_4891_948b_aa269703a45c.slice/crio-a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25 WatchSource:0}: Error finding container a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25: Status 404 returned error can't find the container with id a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25 Oct 30 18:43:48.675264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:48.675230 6492 generic.go:356] "Generic (PLEG): container finished" podID="cd219385-41d8-4891-948b-aa269703a45c" containerID="af4df6487410041feb2d433190a7ba7c22ce538f07c0a4f2fc237d9d22a95156" exitCode=0 Oct 30 18:43:48.675264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:48.675270 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" event={"ID":"cd219385-41d8-4891-948b-aa269703a45c","Type":"ContainerDied","Data":"af4df6487410041feb2d433190a7ba7c22ce538f07c0a4f2fc237d9d22a95156"} Oct 30 18:43:48.675769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:48.675306 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" event={"ID":"cd219385-41d8-4891-948b-aa269703a45c","Type":"ContainerStarted","Data":"a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25"} Oct 30 18:43:49.823323 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.823298 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:49.924947 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.924916 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\") pod \"cd219385-41d8-4891-948b-aa269703a45c\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " Oct 30 18:43:49.925149 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.924960 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\") pod \"cd219385-41d8-4891-948b-aa269703a45c\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " Oct 30 18:43:49.925149 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.925101 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\" (UniqueName: \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\") pod \"cd219385-41d8-4891-948b-aa269703a45c\" (UID: \"cd219385-41d8-4891-948b-aa269703a45c\") " Oct 30 18:43:49.925393 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.925362 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca" pod "cd219385-41d8-4891-948b-aa269703a45c" (UID: "cd219385-41d8-4891-948b-aa269703a45c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:43:49.927142 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.927118 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca" pod "cd219385-41d8-4891-948b-aa269703a45c" (UID: "cd219385-41d8-4891-948b-aa269703a45c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:43:49.927294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:49.927273 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz" pod "cd219385-41d8-4891-948b-aa269703a45c" (UID: "cd219385-41d8-4891-948b-aa269703a45c"). InnerVolumeSpecName "kube-api-access-wdzqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:43:50.026173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.026137 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdzqz\" (UniqueName: \"kubernetes.io/projected/cd219385-41d8-4891-948b-aa269703a45c-kube-api-access-wdzqz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:43:50.026173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.026171 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd219385-41d8-4891-948b-aa269703a45c-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:43:50.026173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.026183 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd219385-41d8-4891-948b-aa269703a45c-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:43:50.687263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.687235 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" Oct 30 18:43:50.687422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.687269 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59c67ac2-5m4ts" event={"ID":"cd219385-41d8-4891-948b-aa269703a45c","Type":"ContainerDied","Data":"a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25"} Oct 30 18:43:50.687422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:43:50.687308 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a81cbc099606ca027bdc8ebe61f5a2b7cf897d661f880435806c03b21d52cd25" Oct 30 18:44:00.989824 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:00.989787 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm"] Oct 30 18:44:00.990260 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:00.990248 6492 cpu_manager.go:414] "containerMap: removing container" podUID="cd219385-41d8-4891-948b-aa269703a45c" containerName="revoke-keys" Oct 30 18:44:00.990320 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:00.990261 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd219385-41d8-4891-948b-aa269703a45c" containerName="revoke-keys" Oct 30 18:44:00.990361 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:00.990335 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd219385-41d8-4891-948b-aa269703a45c" containerName="revoke-keys" Oct 30 18:44:00.994178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:00.994155 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.009365 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.009338 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:44:01.021636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.021615 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:44:01.021636 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.021617 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:44:01.021916 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.021639 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:44:01.025005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.024975 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm"] Oct 30 18:44:01.085631 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.085599 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.085786 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.085637 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89qf\" (UniqueName: \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.085786 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.085757 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.099007 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.098974 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6"] Oct 30 18:44:01.103244 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.103229 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.130716 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.130686 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6"] Oct 30 18:44:01.186608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186572 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89qf\" (UniqueName: \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.186608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186615 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.186853 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186678 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.186853 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186759 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.186931 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186874 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.186972 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.186944 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.187461 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.187442 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.189002 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.188986 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.212629 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.212601 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89qf\" (UniqueName: \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\") pod \"maas-api-revoke-sub-f582a077-m5bjm\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.287880 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.287795 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.287880 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.287849 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.288073 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.287916 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.288669 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.288640 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.290266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.290247 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.308279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.308254 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:01.311575 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.311526 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\") pod \"maas-api-revoke-sub-6991f792-tw6w6\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.416101 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.416065 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:01.609482 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.609452 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm"] Oct 30 18:44:01.611341 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:01.611312 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cba3a53_46df_421e_b546_bc7cef00248a.slice/crio-a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c WatchSource:0}: Error finding container a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c: Status 404 returned error can't find the container with id a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c Oct 30 18:44:01.728647 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.728617 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6"] Oct 30 18:44:01.730696 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:01.730668 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df424f9_f44b_4a86_a0a9_9b5ed0e6f352.slice/crio-4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc WatchSource:0}: Error finding container 4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc: Status 404 returned error can't find the container with id 4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc Oct 30 18:44:01.761767 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.761731 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" event={"ID":"4cba3a53-46df-421e-b546-bc7cef00248a","Type":"ContainerStarted","Data":"a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c"} Oct 30 18:44:01.763163 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:01.763135 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" event={"ID":"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352","Type":"ContainerStarted","Data":"4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc"} Oct 30 18:44:02.563124 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.563088 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq"] Oct 30 18:44:02.570598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.569275 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.586243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.586209 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq"] Oct 30 18:44:02.601015 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.600990 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.601160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.601033 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.601227 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.601172 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrd8r\" (UniqueName: \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.702332 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.702297 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrd8r\" (UniqueName: \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.702504 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.702427 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.702830 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.702599 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.703185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.703162 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.705439 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.705413 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.726519 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.726487 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrd8r\" (UniqueName: \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\") pod \"maas-api-revoke-sub-49f32b28-vf6bq\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:02.771931 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.771893 6492 generic.go:356] "Generic (PLEG): container finished" podID="4cba3a53-46df-421e-b546-bc7cef00248a" containerID="cdda6a0d1e12e2ed23d8ea4c3f3d3d94e6830f6fd94f342d5905c989487ab12b" exitCode=0 Oct 30 18:44:02.772119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.771931 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" event={"ID":"4cba3a53-46df-421e-b546-bc7cef00248a","Type":"ContainerDied","Data":"cdda6a0d1e12e2ed23d8ea4c3f3d3d94e6830f6fd94f342d5905c989487ab12b"} Oct 30 18:44:02.773893 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.773828 6492 generic.go:356] "Generic (PLEG): container finished" podID="1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" containerID="15b31b010363fa6f98397dab1746936ac1236e5967ff2e483aaf75daa2b27ec4" exitCode=0 Oct 30 18:44:02.774000 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.773903 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" event={"ID":"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352","Type":"ContainerDied","Data":"15b31b010363fa6f98397dab1746936ac1236e5967ff2e483aaf75daa2b27ec4"} Oct 30 18:44:02.903078 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:02.903043 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:03.207462 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.207437 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq"] Oct 30 18:44:03.210369 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:03.210326 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5105898_a20a_49bc_ae82_2709062aad76.slice/crio-82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b WatchSource:0}: Error finding container 82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b: Status 404 returned error can't find the container with id 82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b Oct 30 18:44:03.781070 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.780986 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" event={"ID":"b5105898-a20a-49bc-ae82-2709062aad76","Type":"ContainerStarted","Data":"4f92c7cdadfa8b22d2d521470b6fe6bd6feb17d24f888efda3c859eb64c0873d"} Oct 30 18:44:03.781070 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.781041 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" event={"ID":"b5105898-a20a-49bc-ae82-2709062aad76","Type":"ContainerStarted","Data":"82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b"} Oct 30 18:44:03.821965 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.821900 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" podStartSLOduration=1.82188098 podStartE2EDuration="1.82188098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:03.821089859 +0000 UTC m=+715.265480859" watchObservedRunningTime="2026-10-30 18:44:03.82188098 +0000 UTC m=+715.266271978" Oct 30 18:44:03.943963 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.943933 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:03.951921 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:03.951896 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:04.014771 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.014737 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\" (UniqueName: \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\") pod \"4cba3a53-46df-421e-b546-bc7cef00248a\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " Oct 30 18:44:04.014951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.014811 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\") pod \"4cba3a53-46df-421e-b546-bc7cef00248a\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " Oct 30 18:44:04.014951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.014840 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\") pod \"4cba3a53-46df-421e-b546-bc7cef00248a\" (UID: \"4cba3a53-46df-421e-b546-bc7cef00248a\") " Oct 30 18:44:04.014951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.014871 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\") pod \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " Oct 30 18:44:04.014951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.014949 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\") pod \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " Oct 30 18:44:04.015228 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.015199 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca" pod "4cba3a53-46df-421e-b546-bc7cef00248a" (UID: "4cba3a53-46df-421e-b546-bc7cef00248a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:04.015321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.015279 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\") pod \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\" (UID: \"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352\") " Oct 30 18:44:04.015724 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.015689 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca" pod "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" (UID: "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:04.015867 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.015776 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.015867 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.015800 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.017265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.017238 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca" pod "4cba3a53-46df-421e-b546-bc7cef00248a" (UID: "4cba3a53-46df-421e-b546-bc7cef00248a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:04.017469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.017430 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca" pod "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" (UID: "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:04.017810 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.017785 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf" pod "4cba3a53-46df-421e-b546-bc7cef00248a" (UID: "4cba3a53-46df-421e-b546-bc7cef00248a"). InnerVolumeSpecName "kube-api-access-k89qf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:04.018033 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.018007 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk" pod "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" (UID: "1df424f9-f44b-4a86-a0a9-9b5ed0e6f352"). InnerVolumeSpecName "kube-api-access-gs9tk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:04.116933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.116901 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-kube-api-access-gs9tk\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.116933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.116928 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1df424f9-f44b-4a86-a0a9-9b5ed0e6f352-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.117098 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.116938 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k89qf\" (UniqueName: \"kubernetes.io/projected/4cba3a53-46df-421e-b546-bc7cef00248a-kube-api-access-k89qf\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.117098 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.116947 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4cba3a53-46df-421e-b546-bc7cef00248a-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.788435 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.788400 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" Oct 30 18:44:04.788435 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.788412 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f582a077-m5bjm" event={"ID":"4cba3a53-46df-421e-b546-bc7cef00248a","Type":"ContainerDied","Data":"a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c"} Oct 30 18:44:04.788943 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.788474 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a679519a21424f92ba5798ee58a7c3ec5d112008f124bd97d580d5549ddca12c" Oct 30 18:44:04.790180 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.790153 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" Oct 30 18:44:04.790319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.790190 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6991f792-tw6w6" event={"ID":"1df424f9-f44b-4a86-a0a9-9b5ed0e6f352","Type":"ContainerDied","Data":"4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc"} Oct 30 18:44:04.790319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.790237 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4119eb301b241c6e9379cf07ef38045c0ba7370b2098260ac4efebac359787bc" Oct 30 18:44:04.792124 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.792103 6492 generic.go:356] "Generic (PLEG): container finished" podID="b5105898-a20a-49bc-ae82-2709062aad76" containerID="4f92c7cdadfa8b22d2d521470b6fe6bd6feb17d24f888efda3c859eb64c0873d" exitCode=0 Oct 30 18:44:04.792243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:04.792131 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" event={"ID":"b5105898-a20a-49bc-ae82-2709062aad76","Type":"ContainerDied","Data":"4f92c7cdadfa8b22d2d521470b6fe6bd6feb17d24f888efda3c859eb64c0873d"} Oct 30 18:44:05.934556 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:05.934520 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:06.037174 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.037139 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\") pod \"b5105898-a20a-49bc-ae82-2709062aad76\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " Oct 30 18:44:06.037174 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.037177 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\") pod \"b5105898-a20a-49bc-ae82-2709062aad76\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " Oct 30 18:44:06.037390 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.037235 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\" (UniqueName: \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\") pod \"b5105898-a20a-49bc-ae82-2709062aad76\" (UID: \"b5105898-a20a-49bc-ae82-2709062aad76\") " Oct 30 18:44:06.037611 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.037582 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca" pod "b5105898-a20a-49bc-ae82-2709062aad76" (UID: "b5105898-a20a-49bc-ae82-2709062aad76"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:06.039350 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.039321 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca" pod "b5105898-a20a-49bc-ae82-2709062aad76" (UID: "b5105898-a20a-49bc-ae82-2709062aad76"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:06.039451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.039355 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r" pod "b5105898-a20a-49bc-ae82-2709062aad76" (UID: "b5105898-a20a-49bc-ae82-2709062aad76"). InnerVolumeSpecName "kube-api-access-qrd8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:06.138275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.138198 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrd8r\" (UniqueName: \"kubernetes.io/projected/b5105898-a20a-49bc-ae82-2709062aad76-kube-api-access-qrd8r\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:06.138275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.138227 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:06.138275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.138236 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5105898-a20a-49bc-ae82-2709062aad76-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:06.803932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.803899 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" Oct 30 18:44:06.803932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.803925 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49f32b28-vf6bq" event={"ID":"b5105898-a20a-49bc-ae82-2709062aad76","Type":"ContainerDied","Data":"82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b"} Oct 30 18:44:06.804152 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:06.803972 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82236c1c90beb681bd51114e0cbffb7efb73fd9d2894b2ec5e3d7012b838140b" Oct 30 18:44:16.243563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.242784 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x"] Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258046 6492 cpu_manager.go:414] "containerMap: removing container" podUID="4cba3a53-46df-421e-b546-bc7cef00248a" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258075 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cba3a53-46df-421e-b546-bc7cef00248a" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258090 6492 cpu_manager.go:414] "containerMap: removing container" podUID="b5105898-a20a-49bc-ae82-2709062aad76" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258098 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5105898-a20a-49bc-ae82-2709062aad76" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258116 6492 cpu_manager.go:414] "containerMap: removing container" podUID="1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258124 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258235 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cba3a53-46df-421e-b546-bc7cef00248a" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258244 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5105898-a20a-49bc-ae82-2709062aad76" containerName="revoke-keys" Oct 30 18:44:16.261566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.258256 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="1df424f9-f44b-4a86-a0a9-9b5ed0e6f352" containerName="revoke-keys" Oct 30 18:44:16.279558 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.262437 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.289383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.280834 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:44:16.289383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.281012 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:44:16.289383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.281634 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:44:16.289383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.262336 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x"] Oct 30 18:44:16.293133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.293105 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:44:16.357257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.357214 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.357432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.357380 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.357586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.357440 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjdt6\" (UniqueName: \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.458498 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.458455 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.458723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.458566 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.458723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.458633 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjdt6\" (UniqueName: \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.459203 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.459179 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.460883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.460863 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.482764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.482728 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjdt6\" (UniqueName: \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\") pod \"maas-api-revoke-sub-c188a728-76t5x\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.602597 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.602484 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:16.919517 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:16.919492 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x"] Oct 30 18:44:16.922713 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:16.922681 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c6eaee4_f6f0_4c65_9152_e400c9153d15.slice/crio-07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202 WatchSource:0}: Error finding container 07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202: Status 404 returned error can't find the container with id 07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202 Oct 30 18:44:16.977465 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:44:16.977423 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:44:17.875485 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:17.875446 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" event={"ID":"3c6eaee4-f6f0-4c65-9152-e400c9153d15","Type":"ContainerStarted","Data":"8554740c7dfec3bac7bccdae868a4f84c51df106ac461d6403c209d5121d8e3d"} Oct 30 18:44:17.876021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:17.875502 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" event={"ID":"3c6eaee4-f6f0-4c65-9152-e400c9153d15","Type":"ContainerStarted","Data":"07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202"} Oct 30 18:44:17.916486 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:17.916407 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" podStartSLOduration=1.916393367 podStartE2EDuration="1.916393367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:17.915085532 +0000 UTC m=+729.359476531" watchObservedRunningTime="2026-10-30 18:44:17.916393367 +0000 UTC m=+729.360784363" Oct 30 18:44:18.882053 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:18.882020 6492 generic.go:356] "Generic (PLEG): container finished" podID="3c6eaee4-f6f0-4c65-9152-e400c9153d15" containerID="8554740c7dfec3bac7bccdae868a4f84c51df106ac461d6403c209d5121d8e3d" exitCode=0 Oct 30 18:44:18.882605 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:18.882087 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" event={"ID":"3c6eaee4-f6f0-4c65-9152-e400c9153d15","Type":"ContainerDied","Data":"8554740c7dfec3bac7bccdae868a4f84c51df106ac461d6403c209d5121d8e3d"} Oct 30 18:44:19.941140 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:19.941097 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:44:19.945694 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:19.945632 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:19.962263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:19.962236 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-a47385-openshift-default-dockercfg-p98h9\"" Oct 30 18:44:19.978554 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:19.978505 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:44:20.004326 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.004285 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc"] Oct 30 18:44:20.008171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.008147 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.030192 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.030165 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc"] Oct 30 18:44:20.050376 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.050344 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:20.092919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.092803 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.092919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.092852 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.092919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.092889 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.093193 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093130 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093187 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093301 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093254 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhd4t\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093316 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093409 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093354 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093463 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093409 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9htc\" (UniqueName: \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.093463 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093430 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093463 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093453 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.093638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.093492 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.194932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.194851 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\") pod \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " Oct 30 18:44:20.194932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.194911 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\") pod \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " Oct 30 18:44:20.195121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.194996 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\" (UniqueName: \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\") pod \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\" (UID: \"3c6eaee4-f6f0-4c65-9152-e400c9153d15\") " Oct 30 18:44:20.195174 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195156 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195216 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9htc\" (UniqueName: \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.195278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195248 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195278 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195312 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.195325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195309 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca" pod "3c6eaee4-f6f0-4c65-9152-e400c9153d15" (UID: "3c6eaee4-f6f0-4c65-9152-e400c9153d15"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:20.195478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195382 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195425 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195461 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.195678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195528 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195620 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195674 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195710 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhd4t\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195753 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.195953 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195812 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:20.195953 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.195888 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.196589 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.196071 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.196589 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.196222 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.196589 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.196384 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.196785 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.196603 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.197837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.197817 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.198534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.198511 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.198734 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.198711 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.198922 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.198896 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6" pod "3c6eaee4-f6f0-4c65-9152-e400c9153d15" (UID: "3c6eaee4-f6f0-4c65-9152-e400c9153d15"). InnerVolumeSpecName "kube-api-access-jjdt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:20.199529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.199491 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca" pod "3c6eaee4-f6f0-4c65-9152-e400c9153d15" (UID: "3c6eaee4-f6f0-4c65-9152-e400c9153d15"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:20.219205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.219176 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9htc\" (UniqueName: \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\") pod \"maas-api-revoke-sub-504b60eb-xkdwc\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.221738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.221714 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.221864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.221843 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhd4t\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.267345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.267312 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:20.296565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.296517 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c6eaee4-f6f0-4c65-9152-e400c9153d15-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:20.296701 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.296569 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjdt6\" (UniqueName: \"kubernetes.io/projected/3c6eaee4-f6f0-4c65-9152-e400c9153d15-kube-api-access-jjdt6\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:20.326054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.325820 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:20.600749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.600716 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:44:20.602824 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:20.602790 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08925c8d_f8a6_4b35_bc06_3329113b654a.slice/crio-2eef5eded4fb7d67532e34ee9a17789671fa881ce7a0365c40d739041d5d7374 WatchSource:0}: Error finding container 2eef5eded4fb7d67532e34ee9a17789671fa881ce7a0365c40d739041d5d7374: Status 404 returned error can't find the container with id 2eef5eded4fb7d67532e34ee9a17789671fa881ce7a0365c40d739041d5d7374 Oct 30 18:44:20.605216 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.605183 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:20.605302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.605250 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:20.605302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.605279 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:20.647911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.647890 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc"] Oct 30 18:44:20.650433 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:20.650409 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63a7d71_9899_4408_810c_7d3fd1750c44.slice/crio-920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61 WatchSource:0}: Error finding container 920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61: Status 404 returned error can't find the container with id 920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61 Oct 30 18:44:20.895021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.894941 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" Oct 30 18:44:20.895021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.894954 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c188a728-76t5x" event={"ID":"3c6eaee4-f6f0-4c65-9152-e400c9153d15","Type":"ContainerDied","Data":"07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202"} Oct 30 18:44:20.895021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.895011 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07413af9d9b84bf3b940cc60ecb850bb5c77b6141a1b553b3514853570227202" Oct 30 18:44:20.898215 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.896992 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" event={"ID":"08925c8d-f8a6-4b35-bc06-3329113b654a","Type":"ContainerStarted","Data":"c80b8f5c2541f6972c73c5fab5f2e891c22f0467d56fb3a0f44cf7df573701b8"} Oct 30 18:44:20.898215 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.897035 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" event={"ID":"08925c8d-f8a6-4b35-bc06-3329113b654a","Type":"ContainerStarted","Data":"2eef5eded4fb7d67532e34ee9a17789671fa881ce7a0365c40d739041d5d7374"} Oct 30 18:44:20.898501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.898455 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" event={"ID":"c63a7d71-9899-4408-810c-7d3fd1750c44","Type":"ContainerStarted","Data":"920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61"} Oct 30 18:44:20.948243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:20.948174 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podStartSLOduration=1.9481572360000001 podStartE2EDuration="1.948157236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:20.946602398 +0000 UTC m=+732.390993395" watchObservedRunningTime="2026-10-30 18:44:20.948157236 +0000 UTC m=+732.392548233" Oct 30 18:44:21.268319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:21.268282 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:21.269933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:21.269905 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:21.270077 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:21.269952 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:21.921668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:21.918855 6492 generic.go:356] "Generic (PLEG): container finished" podID="c63a7d71-9899-4408-810c-7d3fd1750c44" containerID="e0e4dc003d7c4701d7c431797be749affa59e9aeb04045a817b1b447ccd17310" exitCode=0 Oct 30 18:44:21.921668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:21.920004 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" event={"ID":"c63a7d71-9899-4408-810c-7d3fd1750c44","Type":"ContainerDied","Data":"e0e4dc003d7c4701d7c431797be749affa59e9aeb04045a817b1b447ccd17310"} Oct 30 18:44:22.268116 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:22.268076 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:22.268552 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:22.268132 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:23.094370 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.094344 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:23.139768 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.139737 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\") pod \"c63a7d71-9899-4408-810c-7d3fd1750c44\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " Oct 30 18:44:23.139970 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.139793 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\" (UniqueName: \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\") pod \"c63a7d71-9899-4408-810c-7d3fd1750c44\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " Oct 30 18:44:23.139970 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.139871 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\") pod \"c63a7d71-9899-4408-810c-7d3fd1750c44\" (UID: \"c63a7d71-9899-4408-810c-7d3fd1750c44\") " Oct 30 18:44:23.140151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.140127 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca" pod "c63a7d71-9899-4408-810c-7d3fd1750c44" (UID: "c63a7d71-9899-4408-810c-7d3fd1750c44"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:23.141980 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.141956 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc" pod "c63a7d71-9899-4408-810c-7d3fd1750c44" (UID: "c63a7d71-9899-4408-810c-7d3fd1750c44"). InnerVolumeSpecName "kube-api-access-v9htc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:23.142090 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.141987 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca" pod "c63a7d71-9899-4408-810c-7d3fd1750c44" (UID: "c63a7d71-9899-4408-810c-7d3fd1750c44"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:23.241842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.241800 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9htc\" (UniqueName: \"kubernetes.io/projected/c63a7d71-9899-4408-810c-7d3fd1750c44-kube-api-access-v9htc\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:23.241842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.241834 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:23.241842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.241851 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c63a7d71-9899-4408-810c-7d3fd1750c44-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:23.267691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.267657 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:23.267864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.267706 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:23.934648 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.934611 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" Oct 30 18:44:23.935099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.934616 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-504b60eb-xkdwc" event={"ID":"c63a7d71-9899-4408-810c-7d3fd1750c44","Type":"ContainerDied","Data":"920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61"} Oct 30 18:44:23.935099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:23.934765 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="920f4aa92dedd79e58029530ce61b7efe4fd0038e367e3fdcda8f4792bed2c61" Oct 30 18:44:24.268349 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:24.268308 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:24.268528 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:24.268361 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:25.267988 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:25.267946 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:25.268437 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:25.268004 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:26.267740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:26.267697 6492 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Oct 30 18:44:26.267926 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:26.267755 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Oct 30 18:44:27.363644 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:27.363577 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:27.364158 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:27.364129 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:27.364919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:27.364897 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:44:30.359338 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.359285 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl"] Oct 30 18:44:30.360622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.360594 6492 cpu_manager.go:414] "containerMap: removing container" podUID="3c6eaee4-f6f0-4c65-9152-e400c9153d15" containerName="revoke-keys" Oct 30 18:44:30.360791 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.360774 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c6eaee4-f6f0-4c65-9152-e400c9153d15" containerName="revoke-keys" Oct 30 18:44:30.361031 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.361014 6492 cpu_manager.go:414] "containerMap: removing container" podUID="c63a7d71-9899-4408-810c-7d3fd1750c44" containerName="revoke-keys" Oct 30 18:44:30.361142 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.361129 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c63a7d71-9899-4408-810c-7d3fd1750c44" containerName="revoke-keys" Oct 30 18:44:30.361363 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.361348 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="c63a7d71-9899-4408-810c-7d3fd1750c44" containerName="revoke-keys" Oct 30 18:44:30.361459 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.361448 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c6eaee4-f6f0-4c65-9152-e400c9153d15" containerName="revoke-keys" Oct 30 18:44:30.367937 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.367900 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.382860 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.382829 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl"] Oct 30 18:44:30.392314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.392283 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:44:30.392494 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.392334 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:44:30.392494 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.392337 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:44:30.395048 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.395030 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:44:30.428021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.427719 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4srtb\" (UniqueName: \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.428021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.427872 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.428021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.427900 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.529669 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.528904 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4srtb\" (UniqueName: \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.529669 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.529057 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.529669 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.529084 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.531155 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.531126 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.534178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.534128 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.564720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.564634 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4srtb\" (UniqueName: \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\") pod \"maas-api-revoke-sub-1085ab69-8h6jl\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:30.684095 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:30.684011 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:31.051350 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:31.051316 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl"] Oct 30 18:44:31.053155 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:31.053099 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94888d6_adbd_4f7e_be24_6e6bd89d94c9.slice/crio-0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a WatchSource:0}: Error finding container 0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a: Status 404 returned error can't find the container with id 0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a Oct 30 18:44:32.014190 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:32.014142 6492 generic.go:356] "Generic (PLEG): container finished" podID="b94888d6-adbd-4f7e-be24-6e6bd89d94c9" containerID="80ca6d27d82da6a5dd92f122367c4cd7d82864b37884e6fde8375c9081d960dc" exitCode=0 Oct 30 18:44:32.014648 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:32.014311 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" event={"ID":"b94888d6-adbd-4f7e-be24-6e6bd89d94c9","Type":"ContainerDied","Data":"80ca6d27d82da6a5dd92f122367c4cd7d82864b37884e6fde8375c9081d960dc"} Oct 30 18:44:32.014648 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:32.014360 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" event={"ID":"b94888d6-adbd-4f7e-be24-6e6bd89d94c9","Type":"ContainerStarted","Data":"0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a"} Oct 30 18:44:33.172618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.172567 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:33.257146 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.257113 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\") pod \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " Oct 30 18:44:33.257314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.257160 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\" (UniqueName: \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\") pod \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " Oct 30 18:44:33.257314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.257268 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\") pod \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\" (UID: \"b94888d6-adbd-4f7e-be24-6e6bd89d94c9\") " Oct 30 18:44:33.257565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.257506 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca" pod "b94888d6-adbd-4f7e-be24-6e6bd89d94c9" (UID: "b94888d6-adbd-4f7e-be24-6e6bd89d94c9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:33.257794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.257772 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:33.259402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.259375 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca" pod "b94888d6-adbd-4f7e-be24-6e6bd89d94c9" (UID: "b94888d6-adbd-4f7e-be24-6e6bd89d94c9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:33.259402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.259381 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb" pod "b94888d6-adbd-4f7e-be24-6e6bd89d94c9" (UID: "b94888d6-adbd-4f7e-be24-6e6bd89d94c9"). InnerVolumeSpecName "kube-api-access-4srtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:33.358934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.358845 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4srtb\" (UniqueName: \"kubernetes.io/projected/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-kube-api-access-4srtb\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:33.358934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:33.358877 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b94888d6-adbd-4f7e-be24-6e6bd89d94c9-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:44:34.030744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:34.030716 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" Oct 30 18:44:34.030895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:34.030743 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1085ab69-8h6jl" event={"ID":"b94888d6-adbd-4f7e-be24-6e6bd89d94c9","Type":"ContainerDied","Data":"0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a"} Oct 30 18:44:34.030895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:34.030798 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cf65514af35abf6d9c69248a493b4f1f90c9348ba0bf855cce36080309b153a" Oct 30 18:44:36.164721 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.164683 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:44:36.175666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.173220 6492 cpu_manager.go:414] "containerMap: removing container" podUID="b94888d6-adbd-4f7e-be24-6e6bd89d94c9" containerName="revoke-keys" Oct 30 18:44:36.175666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.173248 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b94888d6-adbd-4f7e-be24-6e6bd89d94c9" containerName="revoke-keys" Oct 30 18:44:36.175666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.173383 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="b94888d6-adbd-4f7e-be24-6e6bd89d94c9" containerName="revoke-keys" Oct 30 18:44:36.177938 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.177656 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.186907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.186879 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:44:36.194253 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.194233 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-b2ef42-openshift-default-dockercfg-m5wtg\"" Oct 30 18:44:36.291009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.290968 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291010 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291245 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291028 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvjms\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291245 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291157 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291245 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291216 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291390 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291276 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291390 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291319 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291488 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291417 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.291488 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.291442 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.392653 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392616 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.392845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392668 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.392845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392713 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.392845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392734 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.392845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392763 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.392908 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393093 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393035 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvjms\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393086 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393100 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393092 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393284 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393211 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393284 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393264 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393434 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.393883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.393862 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.395089 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.395059 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.395383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.395364 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.424663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.422535 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvjms\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.427104 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.427046 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.498490 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.498199 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:36.864134 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.864102 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:44:36.873330 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.873293 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:36.873402 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.873385 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:36.873443 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:36.873429 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:37.066619 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.064515 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" event={"ID":"4c0902b1-64b3-4fae-a058-8adb0c1fc111","Type":"ContainerStarted","Data":"a2635bd8cdc53b0bd495ea7bdc005543d11c1a5030b7b28021583695b5874b09"} Oct 30 18:44:37.066619 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.065555 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" event={"ID":"4c0902b1-64b3-4fae-a058-8adb0c1fc111","Type":"ContainerStarted","Data":"5ea0e3def979ed2cc94765839607c419b1bc5953b8d4fb2348b3d5daaffe661a"} Oct 30 18:44:37.116127 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.114954 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podStartSLOduration=1.114934013 podStartE2EDuration="1.114934013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:37.114358967 +0000 UTC m=+748.558749987" watchObservedRunningTime="2026-10-30 18:44:37.114934013 +0000 UTC m=+748.559325010" Oct 30 18:44:37.485705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.485123 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:44:37.485705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.485197 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:44:37.500439 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.500361 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:37.503016 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.502896 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:37.503016 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.502949 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:37.686496 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.686460 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:44:37.690817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.690792 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.714560 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.712804 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f6f7d5-openshift-default-dockercfg-9sw8n\"" Oct 30 18:44:37.765658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.733077 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:44:37.884686 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.884621 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvfp8\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887039 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887100 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887192 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887224 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887252 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887294 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887337 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.888189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.887387 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988270 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988313 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988335 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988358 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988383 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988417 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988495 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvfp8\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988571 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988602 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988911 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.988996 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.988938 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.989030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.989007 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.989061 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.989041 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.989529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.989506 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.990722 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.990702 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:37.990998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:37.990982 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:38.025317 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.025238 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:38.026218 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.026193 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvfp8\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:38.041880 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.041854 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:38.368270 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.368238 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:44:38.369848 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:38.369800 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66bbe00_2144_4e59_9b40_de0e73000557.slice/crio-e14ecb26f25afa029ae5c4eb9ef44e168536cf932edb82e26db1a3be39bca9f3 WatchSource:0}: Error finding container e14ecb26f25afa029ae5c4eb9ef44e168536cf932edb82e26db1a3be39bca9f3: Status 404 returned error can't find the container with id e14ecb26f25afa029ae5c4eb9ef44e168536cf932edb82e26db1a3be39bca9f3 Oct 30 18:44:38.372516 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.372484 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:38.372644 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.372586 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:38.372644 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.372617 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:38.501563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.500918 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:38.501563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.500981 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:38.848478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.846435 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx"] Oct 30 18:44:38.850951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.850928 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:38.954281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:38.954225 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx"] Oct 30 18:44:39.044309 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.044253 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.044617 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.044594 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.044784 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.044764 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.044931 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.044915 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.045167 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.045148 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.045294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.045278 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg4wz\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-kube-api-access-tg4wz\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.045511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.045492 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.045667 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.045648 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e64212-44f3-44fb-82f3-6f900a72084d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.045880 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.045860 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.085425 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.085389 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" event={"ID":"e66bbe00-2144-4e59-9b40-de0e73000557","Type":"ContainerStarted","Data":"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58"} Oct 30 18:44:39.085607 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.085444 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" event={"ID":"e66bbe00-2144-4e59-9b40-de0e73000557","Type":"ContainerStarted","Data":"e14ecb26f25afa029ae5c4eb9ef44e168536cf932edb82e26db1a3be39bca9f3"} Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.146980 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147032 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg4wz\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-kube-api-access-tg4wz\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147071 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147101 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e64212-44f3-44fb-82f3-6f900a72084d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147143 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147193 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147335 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147364 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.147442 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.148740 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.148976 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.149023 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7e64212-44f3-44fb-82f3-6f900a72084d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.149212 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.150608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.150153 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.155121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.155082 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.155591 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.155526 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.178596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.177401 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podStartSLOduration=2.177379026 podStartE2EDuration="2.177379026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:39.171915477 +0000 UTC m=+750.616306508" watchObservedRunningTime="2026-10-30 18:44:39.177379026 +0000 UTC m=+750.621770023" Oct 30 18:44:39.179374 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.179346 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.181739 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.181711 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg4wz\" (UniqueName: \"kubernetes.io/projected/c7e64212-44f3-44fb-82f3-6f900a72084d-kube-api-access-tg4wz\") pod \"maas-default-gateway-openshift-default-6dcf757899-wltdx\" (UID: \"c7e64212-44f3-44fb-82f3-6f900a72084d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.474101 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.474065 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:39.504586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.501948 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:39.504586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.502016 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:39.863558 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.863442 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx"] Oct 30 18:44:39.865873 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:44:39.865833 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e64212_44f3_44fb_82f3_6f900a72084d.slice/crio-6bb7f75940c0e5d847e454eaed4d7091c813a865cc1fc02eb5fce061696ac70e WatchSource:0}: Error finding container 6bb7f75940c0e5d847e454eaed4d7091c813a865cc1fc02eb5fce061696ac70e: Status 404 returned error can't find the container with id 6bb7f75940c0e5d847e454eaed4d7091c813a865cc1fc02eb5fce061696ac70e Oct 30 18:44:39.870529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.869231 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.870529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.869312 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.870529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:39.869356 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:40.051561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.048652 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:40.051561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.048795 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:40.051561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.048840 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:40.119634 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.115927 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" event={"ID":"c7e64212-44f3-44fb-82f3-6f900a72084d","Type":"ContainerStarted","Data":"86d9a333c3ba8c82e31c7582d6d8c31b155b0301c22bc54431e576ce5fcb62e5"} Oct 30 18:44:40.119634 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.115987 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" event={"ID":"c7e64212-44f3-44fb-82f3-6f900a72084d","Type":"ContainerStarted","Data":"6bb7f75940c0e5d847e454eaed4d7091c813a865cc1fc02eb5fce061696ac70e"} Oct 30 18:44:40.166853 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.166592 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podStartSLOduration=2.166572753 podStartE2EDuration="2.166572753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.165686891 +0000 UTC m=+751.610077888" watchObservedRunningTime="2026-10-30 18:44:40.166572753 +0000 UTC m=+751.610963750" Oct 30 18:44:40.475004 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.474965 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:40.476989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.476963 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:40.477161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.477139 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:40.511566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.507902 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:40.511566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:40.507963 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:41.048566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.047705 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.048566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.047775 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:41.494586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.489458 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.494586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.489520 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:41.503567 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.500905 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.503567 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:41.500965 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:42.043566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.043184 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.043566 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.043251 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:42.478063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.478023 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.478390 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.478345 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:42.500651 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.500611 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.500894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:42.500859 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:43.058578 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.056318 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.058578 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.056385 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:43.475164 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.475128 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.475349 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.475190 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:43.500987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.500945 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.501242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:43.501212 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Oct 30 18:44:44.043405 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.043359 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.043744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.043708 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:44.475590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.474962 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.475590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.475025 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:44.543564 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.542928 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:44.544170 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.544021 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:44.554807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:44.554780 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:44:45.043197 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:45.043162 6492 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.043414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:45.043226 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Oct 30 18:44:45.475383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:45.475339 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.475605 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:45.475415 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:46.047028 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:46.046994 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:46.047412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:46.047383 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:46.048238 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:46.048216 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:44:46.475053 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:46.474999 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:46.475267 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:46.475075 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:47.063481 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:44:47.063400 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:44:47.474723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:47.474688 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:47.475027 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:47.474999 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:48.475172 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:48.475127 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Oct 30 18:44:48.475694 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:48.475211 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Oct 30 18:44:50.515813 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:50.515741 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:44:50.516429 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:50.515855 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:44:51.556735 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:51.556690 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:44:51.557213 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:51.556769 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:44:51.626207 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:51.626172 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:51.626529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:51.626490 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:44:51.627384 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:44:51.627365 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" Oct 30 18:45:00.179302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.179267 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:00.183682 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.183664 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.196664 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.196635 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:00.200491 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.200474 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:00.200656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.200634 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.204207 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.204187 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.206277 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.206255 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:45:00.211449 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.211414 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:45:00.217065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.217043 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:45:00.220059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220029 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.220247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220225 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkzq5\" (UniqueName: \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.220407 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220391 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.220532 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220516 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.220698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220681 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sppn\" (UniqueName: \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.220803 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.220789 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8zq7\" (UniqueName: \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.224911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.224041 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:00.230150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.230127 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:00.235737 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.235707 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.237685 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.237597 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:00.239848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.239828 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:00.257396 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.257358 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:00.321574 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321512 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2mz\" (UniqueName: \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.321770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321647 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.321770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321738 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.321957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321774 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.321957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321836 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sppn\" (UniqueName: \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.321957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.321868 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8zq7\" (UniqueName: \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.322383 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.322362 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.322535 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.322399 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkzq5\" (UniqueName: \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.322535 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.322423 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.322535 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.322490 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.322983 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.322965 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.347214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.347181 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sppn\" (UniqueName: \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\") pod \"maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.347658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.347639 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkzq5\" (UniqueName: \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\") pod \"maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.347710 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.347663 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8zq7\" (UniqueName: \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\") pod \"maas-api-key-cleanup-29889765-jpzp5\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.423367 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.423324 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2mz\" (UniqueName: \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.423599 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.423402 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.424352 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.424323 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.447387 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.447311 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2mz\" (UniqueName: \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\") pod \"maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.497365 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.497321 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:00.537755 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.537700 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:00.540741 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.540714 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:00.552166 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.551964 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:00.861471 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.861434 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:00.862668 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:00.862482 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda765191e_4cd2_499f_877c_3ef46a81a4ee.slice/crio-1e1fdde783920315542d3f56dbb05d3ecc7a6ccbd05cf88439db8273241e6105 WatchSource:0}: Error finding container 1e1fdde783920315542d3f56dbb05d3ecc7a6ccbd05cf88439db8273241e6105: Status 404 returned error can't find the container with id 1e1fdde783920315542d3f56dbb05d3ecc7a6ccbd05cf88439db8273241e6105 Oct 30 18:45:00.913709 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.913674 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:00.927882 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.926589 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:00.933627 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:00.932147 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:00.941334 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:00.941268 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d9be56_2517_4546_85f1_9e18a82bbbc0.slice/crio-06d83d2e50170be2db74c9db4100dc004201eb50a11423f21d352280cfb763a1 WatchSource:0}: Error finding container 06d83d2e50170be2db74c9db4100dc004201eb50a11423f21d352280cfb763a1: Status 404 returned error can't find the container with id 06d83d2e50170be2db74c9db4100dc004201eb50a11423f21d352280cfb763a1 Oct 30 18:45:01.440689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.440583 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerStarted","Data":"5d195dd3b039626173c0f15d1ed4d2acb9a353e485d0ec148f0b27b9ac0ea86f"} Oct 30 18:45:01.440689 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.440645 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerStarted","Data":"1ea68678672165b9b7ac0f4efcf135875c97f25e3a645e2093b24b00dbb5acc6"} Oct 30 18:45:01.442437 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.442412 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerStarted","Data":"21427e0beec030fb743d06ed5affee741f16efaae175a35fee765ec992834b83"} Oct 30 18:45:01.442600 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.442454 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerStarted","Data":"16b3f1314957f597df5e4a54203cbd4aaff7443d3eb9be27cc8b1f488cbd5daa"} Oct 30 18:45:01.444126 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.444105 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerStarted","Data":"1662f25962fb2e3c5073bcbd6f236bde764b60df15669d244d556cf4f78778d2"} Oct 30 18:45:01.444199 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.444143 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerStarted","Data":"1e1fdde783920315542d3f56dbb05d3ecc7a6ccbd05cf88439db8273241e6105"} Oct 30 18:45:01.445698 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.445676 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerStarted","Data":"0a2c9701751fb980a1f6fb289094f595b7dbef62db102ea6f05daeed063735cf"} Oct 30 18:45:01.445781 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.445718 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerStarted","Data":"06d83d2e50170be2db74c9db4100dc004201eb50a11423f21d352280cfb763a1"} Oct 30 18:45:01.487877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.487824 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" podStartSLOduration=1.487811429 podStartE2EDuration="1.487811429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.485773236 +0000 UTC m=+772.930164231" watchObservedRunningTime="2026-10-30 18:45:01.487811429 +0000 UTC m=+772.932202424" Oct 30 18:45:01.528876 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.528823 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" podStartSLOduration=1.528809399 podStartE2EDuration="1.528809399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.527060646 +0000 UTC m=+772.971451641" watchObservedRunningTime="2026-10-30 18:45:01.528809399 +0000 UTC m=+772.973200393" Oct 30 18:45:01.569257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.569203 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" podStartSLOduration=1.569188842 podStartE2EDuration="1.569188842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.566806623 +0000 UTC m=+773.011197642" watchObservedRunningTime="2026-10-30 18:45:01.569188842 +0000 UTC m=+773.013579836" Oct 30 18:45:01.612733 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:01.612680 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" podStartSLOduration=1.612666553 podStartE2EDuration="1.612666553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.608985081 +0000 UTC m=+773.053376095" watchObservedRunningTime="2026-10-30 18:45:01.612666553 +0000 UTC m=+773.057057544" Oct 30 18:45:02.454469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.454435 6492 generic.go:356] "Generic (PLEG): container finished" podID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerID="21427e0beec030fb743d06ed5affee741f16efaae175a35fee765ec992834b83" exitCode=60 Oct 30 18:45:02.454986 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.454550 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerDied","Data":"21427e0beec030fb743d06ed5affee741f16efaae175a35fee765ec992834b83"} Oct 30 18:45:02.454986 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.454896 6492 scope.go:122] "RemoveContainer" containerID="21427e0beec030fb743d06ed5affee741f16efaae175a35fee765ec992834b83" Oct 30 18:45:02.456819 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.456784 6492 generic.go:356] "Generic (PLEG): container finished" podID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerID="1662f25962fb2e3c5073bcbd6f236bde764b60df15669d244d556cf4f78778d2" exitCode=60 Oct 30 18:45:02.456920 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.456872 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerDied","Data":"1662f25962fb2e3c5073bcbd6f236bde764b60df15669d244d556cf4f78778d2"} Oct 30 18:45:02.457101 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.457085 6492 scope.go:122] "RemoveContainer" containerID="1662f25962fb2e3c5073bcbd6f236bde764b60df15669d244d556cf4f78778d2" Oct 30 18:45:02.458678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.458655 6492 generic.go:356] "Generic (PLEG): container finished" podID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerID="0a2c9701751fb980a1f6fb289094f595b7dbef62db102ea6f05daeed063735cf" exitCode=60 Oct 30 18:45:02.458772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.458683 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerDied","Data":"0a2c9701751fb980a1f6fb289094f595b7dbef62db102ea6f05daeed063735cf"} Oct 30 18:45:02.458972 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.458943 6492 scope.go:122] "RemoveContainer" containerID="0a2c9701751fb980a1f6fb289094f595b7dbef62db102ea6f05daeed063735cf" Oct 30 18:45:02.460714 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.460693 6492 generic.go:356] "Generic (PLEG): container finished" podID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerID="5d195dd3b039626173c0f15d1ed4d2acb9a353e485d0ec148f0b27b9ac0ea86f" exitCode=60 Oct 30 18:45:02.460810 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.460753 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerDied","Data":"5d195dd3b039626173c0f15d1ed4d2acb9a353e485d0ec148f0b27b9ac0ea86f"} Oct 30 18:45:02.460960 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:02.460945 6492 scope.go:122] "RemoveContainer" containerID="5d195dd3b039626173c0f15d1ed4d2acb9a353e485d0ec148f0b27b9ac0ea86f" Oct 30 18:45:03.471514 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.471480 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerStarted","Data":"684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552"} Oct 30 18:45:03.473468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.473433 6492 generic.go:356] "Generic (PLEG): container finished" podID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerID="27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f" exitCode=60 Oct 30 18:45:03.473628 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.473488 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerDied","Data":"27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f"} Oct 30 18:45:03.473628 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.473561 6492 scope.go:122] "RemoveContainer" containerID="1662f25962fb2e3c5073bcbd6f236bde764b60df15669d244d556cf4f78778d2" Oct 30 18:45:03.473797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.473776 6492 scope.go:122] "RemoveContainer" containerID="27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f" Oct 30 18:45:03.474035 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:03.474011 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-jpzp5_odh-ai-gateway-infra(a765191e-4cd2-499f-877c-3ef46a81a4ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" Oct 30 18:45:03.475756 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.475737 6492 generic.go:356] "Generic (PLEG): container finished" podID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerID="37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa" exitCode=60 Oct 30 18:45:03.475844 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.475816 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerDied","Data":"37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa"} Oct 30 18:45:03.476079 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.476063 6492 scope.go:122] "RemoveContainer" containerID="37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa" Oct 30 18:45:03.476291 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:03.476271 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg_odh-ai-gateway-infra(10d9be56-2517-4546-85f1-9e18a82bbbc0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" Oct 30 18:45:03.477715 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.477693 6492 generic.go:356] "Generic (PLEG): container finished" podID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerID="ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a" exitCode=60 Oct 30 18:45:03.477813 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.477722 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerDied","Data":"ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a"} Oct 30 18:45:03.477933 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.477912 6492 scope.go:122] "RemoveContainer" containerID="ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a" Oct 30 18:45:03.478086 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:03.478068 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn_odh-ai-gateway-infra(ee8f3626-2876-4be8-b63a-64bdaf1c9a67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" Oct 30 18:45:03.577253 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.577214 6492 scope.go:122] "RemoveContainer" containerID="0a2c9701751fb980a1f6fb289094f595b7dbef62db102ea6f05daeed063735cf" Oct 30 18:45:03.592072 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:03.592047 6492 scope.go:122] "RemoveContainer" containerID="5d195dd3b039626173c0f15d1ed4d2acb9a353e485d0ec148f0b27b9ac0ea86f" Oct 30 18:45:04.486322 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.486298 6492 scope.go:122] "RemoveContainer" containerID="27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f" Oct 30 18:45:04.486823 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:04.486600 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-jpzp5_odh-ai-gateway-infra(a765191e-4cd2-499f-877c-3ef46a81a4ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" Oct 30 18:45:04.487976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.487952 6492 scope.go:122] "RemoveContainer" containerID="37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa" Oct 30 18:45:04.488218 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:04.488190 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg_odh-ai-gateway-infra(10d9be56-2517-4546-85f1-9e18a82bbbc0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" Oct 30 18:45:04.489569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.489531 6492 scope.go:122] "RemoveContainer" containerID="ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a" Oct 30 18:45:04.489783 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:04.489762 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn_odh-ai-gateway-infra(ee8f3626-2876-4be8-b63a-64bdaf1c9a67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" Oct 30 18:45:04.491095 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.491068 6492 generic.go:356] "Generic (PLEG): container finished" podID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerID="684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552" exitCode=60 Oct 30 18:45:04.491193 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.491112 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerDied","Data":"684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552"} Oct 30 18:45:04.491193 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.491173 6492 scope.go:122] "RemoveContainer" containerID="21427e0beec030fb743d06ed5affee741f16efaae175a35fee765ec992834b83" Oct 30 18:45:04.491306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:04.491289 6492 scope.go:122] "RemoveContainer" containerID="684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552" Oct 30 18:45:04.491527 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:04.491509 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck_odh-ai-gateway-infra(aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" Oct 30 18:45:05.499829 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:05.499803 6492 scope.go:122] "RemoveContainer" containerID="684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552" Oct 30 18:45:05.500238 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:05.500029 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck_odh-ai-gateway-infra(aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" Oct 30 18:45:14.659135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:14.659102 6492 scope.go:122] "RemoveContainer" containerID="37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa" Oct 30 18:45:14.659649 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:14.659214 6492 scope.go:122] "RemoveContainer" containerID="27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f" Oct 30 18:45:15.620146 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.620115 6492 generic.go:356] "Generic (PLEG): container finished" podID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerID="b88e7604ed706b9f48ee941636595c978b36e3d81093f620707774c85162190a" exitCode=60 Oct 30 18:45:15.620265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.620190 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerDied","Data":"b88e7604ed706b9f48ee941636595c978b36e3d81093f620707774c85162190a"} Oct 30 18:45:15.620265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.620257 6492 scope.go:122] "RemoveContainer" containerID="27fa1154c5ea512f612cee122e474bce6d213069e0b92db92e6b975a33df8f0f" Oct 30 18:45:15.620696 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.620664 6492 scope.go:122] "RemoveContainer" containerID="b88e7604ed706b9f48ee941636595c978b36e3d81093f620707774c85162190a" Oct 30 18:45:15.620992 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:15.620970 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-jpzp5_odh-ai-gateway-infra(a765191e-4cd2-499f-877c-3ef46a81a4ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" Oct 30 18:45:15.624029 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.623989 6492 generic.go:356] "Generic (PLEG): container finished" podID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerID="94e9b8249c01f37f79f34f41eee779a2e1af25c4a3da887d421f1947ac054387" exitCode=60 Oct 30 18:45:15.624145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.624128 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerDied","Data":"94e9b8249c01f37f79f34f41eee779a2e1af25c4a3da887d421f1947ac054387"} Oct 30 18:45:15.624507 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.624490 6492 scope.go:122] "RemoveContainer" containerID="94e9b8249c01f37f79f34f41eee779a2e1af25c4a3da887d421f1947ac054387" Oct 30 18:45:15.627371 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:15.627339 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg_odh-ai-gateway-infra(10d9be56-2517-4546-85f1-9e18a82bbbc0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" Oct 30 18:45:15.659000 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:15.658982 6492 scope.go:122] "RemoveContainer" containerID="37305c046ccdf8a8e30b9f268f4f96568a1feed50d41bd958afb22f43ed7e4fa" Oct 30 18:45:16.658774 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.658746 6492 scope.go:122] "RemoveContainer" containerID="684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552" Oct 30 18:45:16.672672 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.672646 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:16.711455 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.711422 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:16.841579 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.841523 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:16.856688 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.856664 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\" (UniqueName: \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\") pod \"a765191e-4cd2-499f-877c-3ef46a81a4ee\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " Oct 30 18:45:16.856842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.856822 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\") pod \"a765191e-4cd2-499f-877c-3ef46a81a4ee\" (UID: \"a765191e-4cd2-499f-877c-3ef46a81a4ee\") " Oct 30 18:45:16.857197 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.857171 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca" pod "a765191e-4cd2-499f-877c-3ef46a81a4ee" (UID: "a765191e-4cd2-499f-877c-3ef46a81a4ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:16.858622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.858602 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7" pod "a765191e-4cd2-499f-877c-3ef46a81a4ee" (UID: "a765191e-4cd2-499f-877c-3ef46a81a4ee"). InnerVolumeSpecName "kube-api-access-w8zq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:16.884954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.884927 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:16.970122 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.970083 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a765191e-4cd2-499f-877c-3ef46a81a4ee-maas-api-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:16.970122 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:16.970123 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8zq7\" (UniqueName: \"kubernetes.io/projected/a765191e-4cd2-499f-877c-3ef46a81a4ee-kube-api-access-w8zq7\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:16.981692 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:16.981658 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:45:17.070738 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.070697 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\" (UniqueName: \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\") pod \"10d9be56-2517-4546-85f1-9e18a82bbbc0\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " Oct 30 18:45:17.070915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.070886 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\") pod \"10d9be56-2517-4546-85f1-9e18a82bbbc0\" (UID: \"10d9be56-2517-4546-85f1-9e18a82bbbc0\") " Oct 30 18:45:17.071268 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.071230 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca" pod "10d9be56-2517-4546-85f1-9e18a82bbbc0" (UID: "10d9be56-2517-4546-85f1-9e18a82bbbc0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:17.071438 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.071344 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d9be56-2517-4546-85f1-9e18a82bbbc0-maas-api-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:17.073030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.072998 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz" pod "10d9be56-2517-4546-85f1-9e18a82bbbc0" (UID: "10d9be56-2517-4546-85f1-9e18a82bbbc0"). InnerVolumeSpecName "kube-api-access-jd2mz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:17.172288 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.172242 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd2mz\" (UniqueName: \"kubernetes.io/projected/10d9be56-2517-4546-85f1-9e18a82bbbc0-kube-api-access-jd2mz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:17.654625 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.649643 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" event={"ID":"a765191e-4cd2-499f-877c-3ef46a81a4ee","Type":"ContainerDied","Data":"1e1fdde783920315542d3f56dbb05d3ecc7a6ccbd05cf88439db8273241e6105"} Oct 30 18:45:17.654625 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.649745 6492 scope.go:122] "RemoveContainer" containerID="b88e7604ed706b9f48ee941636595c978b36e3d81093f620707774c85162190a" Oct 30 18:45:17.654625 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.649859 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5" Oct 30 18:45:17.656587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.655777 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" Oct 30 18:45:17.656854 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.656764 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg" event={"ID":"10d9be56-2517-4546-85f1-9e18a82bbbc0","Type":"ContainerDied","Data":"06d83d2e50170be2db74c9db4100dc004201eb50a11423f21d352280cfb763a1"} Oct 30 18:45:17.658423 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.658022 6492 scope.go:122] "RemoveContainer" containerID="ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a" Oct 30 18:45:17.667359 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.666440 6492 generic.go:356] "Generic (PLEG): container finished" podID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerID="696f8145c5b98c07568bf806a23cdd67c5021f1738d4f8c617f6a3c9b867c5f9" exitCode=60 Oct 30 18:45:17.667359 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.666643 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerDied","Data":"696f8145c5b98c07568bf806a23cdd67c5021f1738d4f8c617f6a3c9b867c5f9"} Oct 30 18:45:17.667359 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.667044 6492 scope.go:122] "RemoveContainer" containerID="696f8145c5b98c07568bf806a23cdd67c5021f1738d4f8c617f6a3c9b867c5f9" Oct 30 18:45:17.667359 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:17.667303 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck_odh-ai-gateway-infra(aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" Oct 30 18:45:17.680127 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.679972 6492 scope.go:122] "RemoveContainer" containerID="94e9b8249c01f37f79f34f41eee779a2e1af25c4a3da887d421f1947ac054387" Oct 30 18:45:17.710987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.710961 6492 scope.go:122] "RemoveContainer" containerID="684f37bae9d6c6cb98ca9131f8f582337047cd8ed1c14a713f5f5c5cf410a552" Oct 30 18:45:17.767355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.767265 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:17.771816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.771786 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-b2ef42-29889765-g8bbg"] Oct 30 18:45:17.810304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.810262 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:17.822710 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:17.822674 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-jpzp5"] Oct 30 18:45:18.685652 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:18.685608 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" path="/var/lib/kubelet/pods/10d9be56-2517-4546-85f1-9e18a82bbbc0/volumes" Oct 30 18:45:18.686355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:18.686330 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" path="/var/lib/kubelet/pods/a765191e-4cd2-499f-877c-3ef46a81a4ee/volumes" Oct 30 18:45:18.690798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:18.690774 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerStarted","Data":"fe37a70b57899ec2ed29abe245086bdf2b47814843887ea9125987f37f643b34"} Oct 30 18:45:18.772561 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:18.772506 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:19.367624 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.350869 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:19.453565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.453007 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\") pod \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " Oct 30 18:45:19.453565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.453113 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\" (UniqueName: \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\") pod \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\" (UID: \"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc\") " Oct 30 18:45:19.472327 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.470262 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca" pod "aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" (UID: "aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:19.476562 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.474396 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5" pod "aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" (UID: "aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc"). InnerVolumeSpecName "kube-api-access-zkzq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:19.556622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.556466 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-maas-api-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:19.556622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.556517 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkzq5\" (UniqueName: \"kubernetes.io/projected/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc-kube-api-access-zkzq5\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:19.746985 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.746940 6492 generic.go:356] "Generic (PLEG): container finished" podID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerID="fe37a70b57899ec2ed29abe245086bdf2b47814843887ea9125987f37f643b34" exitCode=60 Oct 30 18:45:19.747760 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.747731 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerDied","Data":"fe37a70b57899ec2ed29abe245086bdf2b47814843887ea9125987f37f643b34"} Oct 30 18:45:19.748063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.748031 6492 scope.go:122] "RemoveContainer" containerID="ed6e1e6b10ba695afaa1e21672e4b135a0ca33a6568d7a9269f0b0aaa8b5f20a" Oct 30 18:45:19.748702 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.748682 6492 scope.go:122] "RemoveContainer" containerID="fe37a70b57899ec2ed29abe245086bdf2b47814843887ea9125987f37f643b34" Oct 30 18:45:19.749228 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:19.749207 6492 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn_odh-ai-gateway-infra(ee8f3626-2876-4be8-b63a-64bdaf1c9a67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" Oct 30 18:45:19.762380 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.762288 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:19.769113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.764559 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" event={"ID":"aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc","Type":"ContainerDied","Data":"16b3f1314957f597df5e4a54203cbd4aaff7443d3eb9be27cc8b1f488cbd5daa"} Oct 30 18:45:19.769113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.764678 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck" Oct 30 18:45:19.797574 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.797246 6492 scope.go:122] "RemoveContainer" containerID="696f8145c5b98c07568bf806a23cdd67c5021f1738d4f8c617f6a3c9b867c5f9" Oct 30 18:45:19.863587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.858763 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:19.864431 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:19.863990 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-a47385-29889765-6mvck"] Oct 30 18:45:20.667077 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:20.667044 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" path="/var/lib/kubelet/pods/aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc/volumes" Oct 30 18:45:20.926797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:20.926730 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:21.075921 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.075889 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\" (UniqueName: \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\") pod \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " Oct 30 18:45:21.076068 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.076026 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\") pod \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\" (UID: \"ee8f3626-2876-4be8-b63a-64bdaf1c9a67\") " Oct 30 18:45:21.076341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.076320 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca" pod "ee8f3626-2876-4be8-b63a-64bdaf1c9a67" (UID: "ee8f3626-2876-4be8-b63a-64bdaf1c9a67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:21.077993 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.077968 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn" pod "ee8f3626-2876-4be8-b63a-64bdaf1c9a67" (UID: "ee8f3626-2876-4be8-b63a-64bdaf1c9a67"). InnerVolumeSpecName "kube-api-access-7sppn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:21.177341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.177251 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-maas-api-service-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:21.177341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.177286 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sppn\" (UniqueName: \"kubernetes.io/projected/ee8f3626-2876-4be8-b63a-64bdaf1c9a67-kube-api-access-7sppn\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:21.783925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.783893 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" Oct 30 18:45:21.784114 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.783889 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn" event={"ID":"ee8f3626-2876-4be8-b63a-64bdaf1c9a67","Type":"ContainerDied","Data":"1ea68678672165b9b7ac0f4efcf135875c97f25e3a645e2093b24b00dbb5acc6"} Oct 30 18:45:21.784114 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.784034 6492 scope.go:122] "RemoveContainer" containerID="fe37a70b57899ec2ed29abe245086bdf2b47814843887ea9125987f37f643b34" Oct 30 18:45:21.858090 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.858040 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:21.865496 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:21.865461 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0ddc26-29889765-2vbmn"] Oct 30 18:45:22.666534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.666480 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" path="/var/lib/kubelet/pods/ee8f3626-2876-4be8-b63a-64bdaf1c9a67/volumes" Oct 30 18:45:22.917871 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.917794 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks"] Oct 30 18:45:22.918271 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918259 6492 cpu_manager.go:414] "containerMap: removing container" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918272 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918282 6492 cpu_manager.go:414] "containerMap: removing container" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918288 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918296 6492 cpu_manager.go:414] "containerMap: removing container" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918300 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918312 6492 cpu_manager.go:414] "containerMap: removing container" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918316 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918322 6492 cpu_manager.go:414] "containerMap: removing container" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918326 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918335 6492 cpu_manager.go:414] "containerMap: removing container" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918340 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918347 6492 cpu_manager.go:414] "containerMap: removing container" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918351 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918364 6492 cpu_manager.go:414] "containerMap: removing container" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918368 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918386 6492 cpu_manager.go:414] "containerMap: removing container" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918391 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918478 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918487 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918494 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918499 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918524 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918529 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918534 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918552 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918559 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa963cb6-f4c3-4ef0-a79b-85ce0ede47fc" containerName="cleanup" Oct 30 18:45:22.918557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.918565 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:22.923115 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.923095 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:22.942454 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.942426 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks"] Oct 30 18:45:22.952021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.951995 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:45:22.952021 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.952023 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:45:22.952180 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.952028 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:45:22.954977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.954955 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:45:22.998584 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.998531 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mcb4\" (UniqueName: \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:22.998730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.998659 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:22.998730 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:22.998723 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.100006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.099972 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mcb4\" (UniqueName: \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.100184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.100029 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.100184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.100062 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.100825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.100800 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.102421 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.102396 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.123350 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.123322 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mcb4\" (UniqueName: \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\") pod \"maas-api-revoke-sub-3524ffb7-rdhks\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.235851 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.235815 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:23.541416 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.541390 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks"] Oct 30 18:45:23.543279 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:23.543247 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb286c0_b48e_4c0a_a2e4_445d8c0008fc.slice/crio-6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4 WatchSource:0}: Error finding container 6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4: Status 404 returned error can't find the container with id 6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4 Oct 30 18:45:23.799830 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:23.799745 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" event={"ID":"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc","Type":"ContainerStarted","Data":"6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4"} Oct 30 18:45:24.808340 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:24.808303 6492 generic.go:356] "Generic (PLEG): container finished" podID="1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" containerID="cac87db6757f5e055675a90dce44190c9140de7f5d3aa0898884562fa541948e" exitCode=0 Oct 30 18:45:24.808774 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:24.808387 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" event={"ID":"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc","Type":"ContainerDied","Data":"cac87db6757f5e055675a90dce44190c9140de7f5d3aa0898884562fa541948e"} Oct 30 18:45:25.964520 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:25.964499 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:26.133895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.133805 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\") pod \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " Oct 30 18:45:26.133895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.133875 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\") pod \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " Oct 30 18:45:26.134154 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.133913 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\" (UniqueName: \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\") pod \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\" (UID: \"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc\") " Oct 30 18:45:26.134259 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.134235 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca" pod "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" (UID: "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:26.134329 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.134319 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:26.136102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.136070 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4" pod "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" (UID: "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc"). InnerVolumeSpecName "kube-api-access-4mcb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:26.136217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.136198 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca" pod "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" (UID: "1fb286c0-b48e-4c0a-a2e4-445d8c0008fc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:26.235043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.235012 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:26.235043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.235040 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mcb4\" (UniqueName: \"kubernetes.io/projected/1fb286c0-b48e-4c0a-a2e4-445d8c0008fc-kube-api-access-4mcb4\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:26.822658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.822628 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" Oct 30 18:45:26.822658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.822639 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3524ffb7-rdhks" event={"ID":"1fb286c0-b48e-4c0a-a2e4-445d8c0008fc","Type":"ContainerDied","Data":"6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4"} Oct 30 18:45:26.822875 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:26.822698 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6423a61e6730e22c8be5e05507cd1dd685ea56d8bfd7be423d187595c80058b4" Oct 30 18:45:35.434912 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.434880 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9"] Oct 30 18:45:35.435419 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435405 6492 cpu_manager.go:414] "containerMap: removing container" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435421 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435434 6492 cpu_manager.go:414] "containerMap: removing container" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435439 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10d9be56-2517-4546-85f1-9e18a82bbbc0" containerName="cleanup" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435448 6492 cpu_manager.go:414] "containerMap: removing container" podUID="1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" containerName="revoke-keys" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435455 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" containerName="revoke-keys" Oct 30 18:45:35.435465 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435465 6492 cpu_manager.go:414] "containerMap: removing container" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:35.435668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435471 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:35.435668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435534 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fb286c0-b48e-4c0a-a2e4-445d8c0008fc" containerName="revoke-keys" Oct 30 18:45:35.435668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435566 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee8f3626-2876-4be8-b63a-64bdaf1c9a67" containerName="cleanup" Oct 30 18:45:35.435668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.435574 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="a765191e-4cd2-499f-877c-3ef46a81a4ee" containerName="cleanup" Oct 30 18:45:35.438715 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.438696 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.458120 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.458097 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9"] Oct 30 18:45:35.472993 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.466968 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:45:35.472993 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.467097 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:45:35.472993 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.467164 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:45:35.472993 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.467266 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:45:35.537982 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.537930 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.538156 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.537997 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.538156 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.538129 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrj4w\" (UniqueName: \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.639484 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.639445 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.639693 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.639504 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.639693 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.639612 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrj4w\" (UniqueName: \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.640184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.640163 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.642164 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.642138 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.663576 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.663521 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrj4w\" (UniqueName: \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\") pod \"maas-api-revoke-sub-65d74124-mnsq9\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:35.751963 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:35.751933 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:36.059805 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:36.059761 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9"] Oct 30 18:45:36.064080 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:36.064050 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541dd9ad_1acf_4edf_8a72_6bbe46b28c6c.slice/crio-28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3 WatchSource:0}: Error finding container 28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3: Status 404 returned error can't find the container with id 28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3 Oct 30 18:45:36.892556 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:36.892514 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" event={"ID":"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c","Type":"ContainerStarted","Data":"28dd0e19681f4511704d43ce7892092000645a83ea7bcc0b9c3d4798c8595a41"} Oct 30 18:45:36.892946 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:36.892580 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" event={"ID":"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c","Type":"ContainerStarted","Data":"28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3"} Oct 30 18:45:36.934385 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:36.934325 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" podStartSLOduration=1.9343055630000001 podStartE2EDuration="1.934305563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:36.932423317 +0000 UTC m=+808.376814314" watchObservedRunningTime="2026-10-30 18:45:36.934305563 +0000 UTC m=+808.378696559" Oct 30 18:45:37.902069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:37.902034 6492 generic.go:356] "Generic (PLEG): container finished" podID="541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" containerID="28dd0e19681f4511704d43ce7892092000645a83ea7bcc0b9c3d4798c8595a41" exitCode=0 Oct 30 18:45:37.902531 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:37.902113 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" event={"ID":"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c","Type":"ContainerDied","Data":"28dd0e19681f4511704d43ce7892092000645a83ea7bcc0b9c3d4798c8595a41"} Oct 30 18:45:39.064399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.064376 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:39.180182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.180089 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\") pod \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " Oct 30 18:45:39.180332 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.180195 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\") pod \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " Oct 30 18:45:39.180332 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.180304 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\" (UniqueName: \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\") pod \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\" (UID: \"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c\") " Oct 30 18:45:39.180577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.180553 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca" pod "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" (UID: "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:39.180737 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.180721 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:39.184643 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.182634 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w" pod "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" (UID: "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c"). InnerVolumeSpecName "kube-api-access-mrj4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:39.184643 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.182635 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca" pod "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" (UID: "541dd9ad-1acf-4edf-8a72-6bbe46b28c6c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:39.281529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.281476 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:39.281529 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.281509 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrj4w\" (UniqueName: \"kubernetes.io/projected/541dd9ad-1acf-4edf-8a72-6bbe46b28c6c-kube-api-access-mrj4w\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:39.916092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.916006 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" event={"ID":"541dd9ad-1acf-4edf-8a72-6bbe46b28c6c","Type":"ContainerDied","Data":"28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3"} Oct 30 18:45:39.916092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.916067 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28ebed8961132aa001ca4b0087272b082dd6114094333c5b3810f596ef8556f3" Oct 30 18:45:39.916092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:39.916019 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65d74124-mnsq9" Oct 30 18:45:46.313205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.313162 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd"] Oct 30 18:45:46.313841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.313821 6492 cpu_manager.go:414] "containerMap: removing container" podUID="541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" containerName="revoke-keys" Oct 30 18:45:46.313841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.313840 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" containerName="revoke-keys" Oct 30 18:45:46.313977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.313947 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="541dd9ad-1acf-4edf-8a72-6bbe46b28c6c" containerName="revoke-keys" Oct 30 18:45:46.317840 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.317817 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.333587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.333562 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:45:46.333694 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.333658 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:45:46.352613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.349895 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:45:46.352613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.349938 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:45:46.354662 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.354640 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd"] Oct 30 18:45:46.456478 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.456447 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.456673 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.456500 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.456673 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.456636 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-779ts\" (UniqueName: \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.557468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.557427 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.557692 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.557502 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.557692 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.557584 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-779ts\" (UniqueName: \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.558353 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.558320 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.559940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.559915 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.581633 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.581571 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-779ts\" (UniqueName: \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\") pod \"maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.663614 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.663586 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:46.977240 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:45:46.977177 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:45:46.994279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:46.994245 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd"] Oct 30 18:45:47.002242 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:47.002205 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de70891_5dcc_45b1_858a_4ee86cf49bfc.slice/crio-5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59 WatchSource:0}: Error finding container 5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59: Status 404 returned error can't find the container with id 5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59 Oct 30 18:45:47.271941 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.271853 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:45:47.279113 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.279085 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.301422 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.301350 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:45:47.305451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.305402 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"openshift-service-ca.crt\"" Oct 30 18:45:47.305451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.305401 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"default-dockercfg-vnsmm\"" Oct 30 18:45:47.305679 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.305461 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"kube-root-ca.crt\"" Oct 30 18:45:47.309810 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.309340 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"e2e-conflict-a-6c6935b2-kserve-self-signed-certs\"" Oct 30 18:45:47.365860 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.365821 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.366279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.365875 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.366279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.366029 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgvgh\" (UniqueName: \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.366279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.366162 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.366279 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.366195 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.366515 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.366294 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.467978 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.467919 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468015 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468050 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468103 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgvgh\" (UniqueName: \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468192 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468355 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468217 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468458 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468430 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.468842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468812 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.469008 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.468941 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.470935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.470913 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.471217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.471196 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.500084 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.500027 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgvgh\" (UniqueName: \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\") pod \"e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.596659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.596360 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:47.924306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.924276 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:45:47.925254 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:45:47.925228 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc965d665_eff2_4db5_bdf3_b5513145ddd2.slice/crio-27e2c31d7bfa4ef7c242b7cc0aa2574cd814641daa4c44a3393ce7bf1656e655 WatchSource:0}: Error finding container 27e2c31d7bfa4ef7c242b7cc0aa2574cd814641daa4c44a3393ce7bf1656e655: Status 404 returned error can't find the container with id 27e2c31d7bfa4ef7c242b7cc0aa2574cd814641daa4c44a3393ce7bf1656e655 Oct 30 18:45:47.973737 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.973710 6492 generic.go:356] "Generic (PLEG): container finished" podID="8de70891-5dcc-45b1-858a-4ee86cf49bfc" containerID="53bb41af9be91fc6b06505132c71922816c7fbd80f107bac2af73e63b51ba1d3" exitCode=0 Oct 30 18:45:47.973835 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.973796 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" event={"ID":"8de70891-5dcc-45b1-858a-4ee86cf49bfc","Type":"ContainerDied","Data":"53bb41af9be91fc6b06505132c71922816c7fbd80f107bac2af73e63b51ba1d3"} Oct 30 18:45:47.973889 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.973849 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" event={"ID":"8de70891-5dcc-45b1-858a-4ee86cf49bfc","Type":"ContainerStarted","Data":"5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59"} Oct 30 18:45:47.974926 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:47.974907 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" event={"ID":"c965d665-eff2-4db5-bdf3-b5513145ddd2","Type":"ContainerStarted","Data":"27e2c31d7bfa4ef7c242b7cc0aa2574cd814641daa4c44a3393ce7bf1656e655"} Oct 30 18:45:48.982244 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:48.982156 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" event={"ID":"c965d665-eff2-4db5-bdf3-b5513145ddd2","Type":"ContainerStarted","Data":"36f5f4d378b6846c80e13ef6e496bcb7af0f135150c6373b7e502bc0effb975d"} Oct 30 18:45:49.028604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.028465 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" podStartSLOduration=2.028447489 podStartE2EDuration="2.028447489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:49.027009938 +0000 UTC m=+820.471400932" watchObservedRunningTime="2026-10-30 18:45:49.028447489 +0000 UTC m=+820.472838483" Oct 30 18:45:49.134431 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.134399 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:49.188399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.188365 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\") pod \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " Oct 30 18:45:49.188596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.188523 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\") pod \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " Oct 30 18:45:49.188691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.188674 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\" (UniqueName: \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\") pod \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\" (UID: \"8de70891-5dcc-45b1-858a-4ee86cf49bfc\") " Oct 30 18:45:49.188850 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.188820 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca" pod "8de70891-5dcc-45b1-858a-4ee86cf49bfc" (UID: "8de70891-5dcc-45b1-858a-4ee86cf49bfc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:49.189069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.189046 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:49.190646 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.190622 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts" pod "8de70891-5dcc-45b1-858a-4ee86cf49bfc" (UID: "8de70891-5dcc-45b1-858a-4ee86cf49bfc"). InnerVolumeSpecName "kube-api-access-779ts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:49.190720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.190649 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca" pod "8de70891-5dcc-45b1-858a-4ee86cf49bfc" (UID: "8de70891-5dcc-45b1-858a-4ee86cf49bfc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:49.290618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.290509 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-779ts\" (UniqueName: \"kubernetes.io/projected/8de70891-5dcc-45b1-858a-4ee86cf49bfc-kube-api-access-779ts\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:49.290618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.290554 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8de70891-5dcc-45b1-858a-4ee86cf49bfc-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:45:49.597932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.597853 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:49.991510 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.991480 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" Oct 30 18:45:49.992006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.991518 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd" event={"ID":"8de70891-5dcc-45b1-858a-4ee86cf49bfc","Type":"ContainerDied","Data":"5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59"} Oct 30 18:45:49.992006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:49.991596 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5500e4237ea8b1bb545f6cb95583a96b61807a062f155fba9b0fa91c234b5d59" Oct 30 18:45:50.841777 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:50.841732 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:45:52.490573 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:52.489805 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-2bv6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:45:52.490573 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:52.489880 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2bv6f" podUID="ff54aab4-243f-4c0f-b941-5e2c6839ab77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:45:52.656531 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:52.656476 6492 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wltdx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:45:52.656882 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:52.656848 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wltdx" podUID="c7e64212-44f3-44fb-82f3-6f900a72084d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:45:56.760637 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:56.760602 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:45:56.832337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:56.832293 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:45:57.597801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:57.597767 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:57.611195 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:57.611163 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:45:58.160648 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:45:58.160617 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:46:01.765565 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:01.764625 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" containerID="cri-o://59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8" gracePeriod=25 Oct 30 18:46:01.833563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:01.832610 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" containerID="cri-o://4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310" gracePeriod=25 Oct 30 18:46:03.724274 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.724223 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj"] Oct 30 18:46:03.724811 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.724764 6492 cpu_manager.go:414] "containerMap: removing container" podUID="8de70891-5dcc-45b1-858a-4ee86cf49bfc" containerName="revoke-keys" Oct 30 18:46:03.724811 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.724778 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8de70891-5dcc-45b1-858a-4ee86cf49bfc" containerName="revoke-keys" Oct 30 18:46:03.724943 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.724864 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="8de70891-5dcc-45b1-858a-4ee86cf49bfc" containerName="revoke-keys" Oct 30 18:46:03.728878 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.728857 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.749604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.749581 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj"] Oct 30 18:46:03.753145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.753120 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.753344 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.753322 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2phn8\" (UniqueName: \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.753451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.753382 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.757889 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.757864 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:03.758042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.757877 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:03.758042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.757887 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:03.758042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.757908 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:03.854841 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.854810 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.855059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.854908 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2phn8\" (UniqueName: \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.855059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.854937 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.855480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.855461 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.857150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.857120 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:03.878173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:03.878151 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2phn8\" (UniqueName: \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\") pod \"maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:04.042575 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:04.042454 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:04.347928 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:04.347897 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj"] Oct 30 18:46:04.349758 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:04.349727 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27987f12_1a35_44ec_aa68_d59a5b772e65.slice/crio-9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284 WatchSource:0}: Error finding container 9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284: Status 404 returned error can't find the container with id 9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284 Oct 30 18:46:05.210848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.210811 6492 generic.go:356] "Generic (PLEG): container finished" podID="27987f12-1a35-44ec-aa68-d59a5b772e65" containerID="f202c3da64da5fc19eb33f0b2a2e21cc2320dc29dc32976cd936be66c5aec477" exitCode=0 Oct 30 18:46:05.211263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.210907 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" event={"ID":"27987f12-1a35-44ec-aa68-d59a5b772e65","Type":"ContainerDied","Data":"f202c3da64da5fc19eb33f0b2a2e21cc2320dc29dc32976cd936be66c5aec477"} Oct 30 18:46:05.211263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.210959 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" event={"ID":"27987f12-1a35-44ec-aa68-d59a5b772e65","Type":"ContainerStarted","Data":"9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284"} Oct 30 18:46:05.700489 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.700447 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv"] Oct 30 18:46:05.704603 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.704573 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.727589 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.727559 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv"] Oct 30 18:46:05.778211 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.778177 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpqfz\" (UniqueName: \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.778211 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.778215 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.778427 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.778249 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.871112 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.871077 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" podUID="fae2421f-38e9-4124-a446-75085b4b1da9" containerName="main" containerID="cri-o://312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f" gracePeriod=45 Oct 30 18:46:05.879793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.879766 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpqfz\" (UniqueName: \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.879890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.879804 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.879890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.879857 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.880440 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.880418 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.882004 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.881983 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:05.903444 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:05.903421 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpqfz\" (UniqueName: \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\") pod \"maas-api-revoke-sub-f8c43280-mfscv\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:06.022184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.022146 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:06.144809 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.144446 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:46:06.184007 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.183978 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184019 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184067 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184156 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184205 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\" (UniqueName: \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184339 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184311 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\" (UniqueName: \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\") pod \"fae2421f-38e9-4124-a446-75085b4b1da9\" (UID: \"fae2421f-38e9-4124-a446-75085b4b1da9\") " Oct 30 18:46:06.184396 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184327 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.184396 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184383 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.184522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184489 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.184723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184704 6492 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-tmp-dir\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.184784 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184727 6492 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-home\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.184784 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.184742 6492 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-model-cache\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.186917 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.186619 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.186917 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.186843 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:06.187256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.187217 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6" pod "fae2421f-38e9-4124-a446-75085b4b1da9" (UID: "fae2421f-38e9-4124-a446-75085b4b1da9"). InnerVolumeSpecName "kube-api-access-tlqk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:06.228350 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.228311 6492 generic.go:356] "Generic (PLEG): container finished" podID="fae2421f-38e9-4124-a446-75085b4b1da9" containerID="312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f" exitCode=0 Oct 30 18:46:06.228852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.228608 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" event={"ID":"fae2421f-38e9-4124-a446-75085b4b1da9","Type":"ContainerDied","Data":"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f"} Oct 30 18:46:06.228852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.228629 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" Oct 30 18:46:06.228852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.228648 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95" event={"ID":"fae2421f-38e9-4124-a446-75085b4b1da9","Type":"ContainerDied","Data":"aaa81838571dc0805fb68772469fecad7965c7f246a83100a1f3e2e364bfd5de"} Oct 30 18:46:06.228852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.228670 6492 scope.go:122] "RemoveContainer" containerID="312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f" Oct 30 18:46:06.251386 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.251215 6492 scope.go:122] "RemoveContainer" containerID="312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f" Oct 30 18:46:06.251657 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:06.251621 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f\": container with ID starting with 312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f not found: ID does not exist" containerID="312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f" Oct 30 18:46:06.251769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.251669 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f"} err="failed to get container status \"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f\": rpc error: code = NotFound desc = could not find container \"312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f\": container with ID starting with 312361ee9a1cb9d7dd99bd5721ef8c002f95f3b79123c2ad992d0b45c046070f not found: ID does not exist" Oct 30 18:46:06.286166 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.286136 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlqk6\" (UniqueName: \"kubernetes.io/projected/fae2421f-38e9-4124-a446-75085b4b1da9-kube-api-access-tlqk6\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.286166 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.286160 6492 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fae2421f-38e9-4124-a446-75085b4b1da9-dshm\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.286166 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.286170 6492 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fae2421f-38e9-4124-a446-75085b4b1da9-tls-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.289423 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.289394 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:46:06.294105 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.294080 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/premium-w4-a47385-kserve-c6db4bdbb-j4g95"] Oct 30 18:46:06.345299 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.345260 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv"] Oct 30 18:46:06.348688 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:06.348649 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bfd517_a7cd_4b83_a3e4_a28e53fd2bf4.slice/crio-b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914 WatchSource:0}: Error finding container b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914: Status 404 returned error can't find the container with id b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914 Oct 30 18:46:06.387812 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.387793 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:06.488650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.488566 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\") pod \"27987f12-1a35-44ec-aa68-d59a5b772e65\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " Oct 30 18:46:06.488650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.488612 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\" (UniqueName: \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\") pod \"27987f12-1a35-44ec-aa68-d59a5b772e65\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " Oct 30 18:46:06.488907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.488694 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\") pod \"27987f12-1a35-44ec-aa68-d59a5b772e65\" (UID: \"27987f12-1a35-44ec-aa68-d59a5b772e65\") " Oct 30 18:46:06.488984 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.488959 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca" pod "27987f12-1a35-44ec-aa68-d59a5b772e65" (UID: "27987f12-1a35-44ec-aa68-d59a5b772e65"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:06.489088 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.489067 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.490796 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.490774 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca" pod "27987f12-1a35-44ec-aa68-d59a5b772e65" (UID: "27987f12-1a35-44ec-aa68-d59a5b772e65"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:06.490895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.490876 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8" pod "27987f12-1a35-44ec-aa68-d59a5b772e65" (UID: "27987f12-1a35-44ec-aa68-d59a5b772e65"). InnerVolumeSpecName "kube-api-access-2phn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:06.590281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.590248 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2phn8\" (UniqueName: \"kubernetes.io/projected/27987f12-1a35-44ec-aa68-d59a5b772e65-kube-api-access-2phn8\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.590281 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.590281 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27987f12-1a35-44ec-aa68-d59a5b772e65-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.664726 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.664696 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fae2421f-38e9-4124-a446-75085b4b1da9" path="/var/lib/kubelet/pods/fae2421f-38e9-4124-a446-75085b4b1da9/volumes" Oct 30 18:46:06.911133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.911096 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:06.911133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:06.911126 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:07.237508 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.237472 6492 generic.go:356] "Generic (PLEG): container finished" podID="74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" containerID="30c1b8739c7a34cdff298648a527c21c9397c325a9961bcd091fabb4bd6396a4" exitCode=0 Oct 30 18:46:07.237974 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.237580 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" event={"ID":"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4","Type":"ContainerDied","Data":"30c1b8739c7a34cdff298648a527c21c9397c325a9961bcd091fabb4bd6396a4"} Oct 30 18:46:07.237974 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.237639 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" event={"ID":"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4","Type":"ContainerStarted","Data":"b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914"} Oct 30 18:46:07.240161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.240138 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" event={"ID":"27987f12-1a35-44ec-aa68-d59a5b772e65","Type":"ContainerDied","Data":"9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284"} Oct 30 18:46:07.240161 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.240160 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj" Oct 30 18:46:07.240368 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:07.240171 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f67d2ac2954a1aff3ca104800c2682fc2fefc3c0bd85ce0459063ae3f380284" Oct 30 18:46:08.404241 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.404211 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:08.448241 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.448203 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:46:08.516622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.515941 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\") pod \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " Oct 30 18:46:08.516622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.516007 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\") pod \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " Oct 30 18:46:08.516622 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.516199 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\" (UniqueName: \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\") pod \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\" (UID: \"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4\") " Oct 30 18:46:08.519114 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.519071 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca" pod "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" (UID: "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:08.520184 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.520153 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz" pod "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" (UID: "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4"). InnerVolumeSpecName "kube-api-access-wpqfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:08.521613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.521585 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca" pod "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" (UID: "74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:08.618065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.618022 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:08.618065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.618054 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:08.618065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:08.618065 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpqfz\" (UniqueName: \"kubernetes.io/projected/74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4-kube-api-access-wpqfz\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:09.257304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:09.257263 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" event={"ID":"74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4","Type":"ContainerDied","Data":"b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914"} Oct 30 18:46:09.257486 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:09.257317 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0c6fd2f60b8855bb7787ffbddf8483f51acf0f610d37cda9e4ad2e023d61914" Oct 30 18:46:09.257486 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:09.257280 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8c43280-mfscv" Oct 30 18:46:14.067187 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067149 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf"] Oct 30 18:46:14.067656 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067641 6492 cpu_manager.go:414] "containerMap: removing container" podUID="27987f12-1a35-44ec-aa68-d59a5b772e65" containerName="revoke-keys" Oct 30 18:46:14.067720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067658 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27987f12-1a35-44ec-aa68-d59a5b772e65" containerName="revoke-keys" Oct 30 18:46:14.067720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067675 6492 cpu_manager.go:414] "containerMap: removing container" podUID="fae2421f-38e9-4124-a446-75085b4b1da9" containerName="main" Oct 30 18:46:14.067720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067681 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fae2421f-38e9-4124-a446-75085b4b1da9" containerName="main" Oct 30 18:46:14.067720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067688 6492 cpu_manager.go:414] "containerMap: removing container" podUID="74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" containerName="revoke-keys" Oct 30 18:46:14.067720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067694 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" containerName="revoke-keys" Oct 30 18:46:14.067895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067784 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="27987f12-1a35-44ec-aa68-d59a5b772e65" containerName="revoke-keys" Oct 30 18:46:14.067895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067794 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="fae2421f-38e9-4124-a446-75085b4b1da9" containerName="main" Oct 30 18:46:14.067895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.067800 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4" containerName="revoke-keys" Oct 30 18:46:14.072601 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.072585 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.095128 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.095094 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf"] Oct 30 18:46:14.103799 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.103614 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:14.103799 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.103677 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:14.103799 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.103758 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:14.104030 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.103845 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:14.178186 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.178133 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.178351 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.178311 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xb5x\" (UniqueName: \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.178404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.178358 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.279285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.279253 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xb5x\" (UniqueName: \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.279473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.279303 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.279473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.279353 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.280058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.280033 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.281848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.281827 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.303845 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.303814 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xb5x\" (UniqueName: \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\") pod \"maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.386054 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.385978 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:14.645969 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.645892 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:46:14.707747 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.707693 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:46:14.728301 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:14.728256 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9ae676_4a3f_40f5_8551_8e2316b3a533.slice/crio-1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6 WatchSource:0}: Error finding container 1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6: Status 404 returned error can't find the container with id 1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6 Oct 30 18:46:14.728904 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:14.728879 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf"] Oct 30 18:46:15.306498 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:15.306456 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" event={"ID":"0e9ae676-4a3f-40f5-8551-8e2316b3a533","Type":"ContainerStarted","Data":"405b4bdec929366847d3ec90f0f6ff1c53faaa501add54c7cac3d4282f705758"} Oct 30 18:46:15.307026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:15.306567 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" event={"ID":"0e9ae676-4a3f-40f5-8551-8e2316b3a533","Type":"ContainerStarted","Data":"1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6"} Oct 30 18:46:15.351276 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:15.351223 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" podStartSLOduration=1.351209419 podStartE2EDuration="1.351209419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:15.349919158 +0000 UTC m=+846.794310152" watchObservedRunningTime="2026-10-30 18:46:15.351209419 +0000 UTC m=+846.795600414" Oct 30 18:46:16.314857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:16.314821 6492 generic.go:356] "Generic (PLEG): container finished" podID="0e9ae676-4a3f-40f5-8551-8e2316b3a533" containerID="405b4bdec929366847d3ec90f0f6ff1c53faaa501add54c7cac3d4282f705758" exitCode=0 Oct 30 18:46:16.315256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:16.314905 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" event={"ID":"0e9ae676-4a3f-40f5-8551-8e2316b3a533","Type":"ContainerDied","Data":"405b4bdec929366847d3ec90f0f6ff1c53faaa501add54c7cac3d4282f705758"} Oct 30 18:46:16.911006 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:16.910962 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:16.911171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:16.911043 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:16.974647 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:16.974596 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:46:17.481303 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.481273 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:17.543047 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.543006 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\") pod \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " Oct 30 18:46:17.543266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.543153 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\") pod \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " Oct 30 18:46:17.543266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.543205 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\" (UniqueName: \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\") pod \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\" (UID: \"0e9ae676-4a3f-40f5-8551-8e2316b3a533\") " Oct 30 18:46:17.544666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.544636 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca" pod "0e9ae676-4a3f-40f5-8551-8e2316b3a533" (UID: "0e9ae676-4a3f-40f5-8551-8e2316b3a533"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:17.550865 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.550841 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x" pod "0e9ae676-4a3f-40f5-8551-8e2316b3a533" (UID: "0e9ae676-4a3f-40f5-8551-8e2316b3a533"). InnerVolumeSpecName "kube-api-access-2xb5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:17.551677 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.551648 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca" pod "0e9ae676-4a3f-40f5-8551-8e2316b3a533" (UID: "0e9ae676-4a3f-40f5-8551-8e2316b3a533"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:17.645003 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.644925 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:17.645003 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.644962 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xb5x\" (UniqueName: \"kubernetes.io/projected/0e9ae676-4a3f-40f5-8551-8e2316b3a533-kube-api-access-2xb5x\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:17.645003 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:17.644977 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e9ae676-4a3f-40f5-8551-8e2316b3a533-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:18.333827 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:18.333795 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" Oct 30 18:46:18.334005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:18.333831 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf" event={"ID":"0e9ae676-4a3f-40f5-8551-8e2316b3a533","Type":"ContainerDied","Data":"1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6"} Oct 30 18:46:18.334005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:18.333876 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d8f40217412d46d64e6cdf8aea56193ea0564ae5118264ce0b94f858c4b63b6" Oct 30 18:46:18.775477 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:18.775443 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:46:19.647008 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:19.646912 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" containerID="cri-o://dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18" gracePeriod=25 Oct 30 18:46:19.708977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:19.708922 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" containerID="cri-o://640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd" gracePeriod=25 Oct 30 18:46:20.983159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:20.983116 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:46:20.983696 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:20.983679 6492 cpu_manager.go:414] "containerMap: removing container" podUID="0e9ae676-4a3f-40f5-8551-8e2316b3a533" containerName="revoke-keys" Oct 30 18:46:20.983798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:20.983698 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e9ae676-4a3f-40f5-8551-8e2316b3a533" containerName="revoke-keys" Oct 30 18:46:20.983855 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:20.983799 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e9ae676-4a3f-40f5-8551-8e2316b3a533" containerName="revoke-keys" Oct 30 18:46:20.987192 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:20.987172 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.003020 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.002998 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-0ddc26-openshift-default-dockercfg-sc7gm\"" Oct 30 18:46:21.018076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.018044 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:46:21.083927 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.083891 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084103 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.083940 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084103 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084074 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084200 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084105 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084200 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084133 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084338 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084278 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084391 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084472 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084459 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6fbv\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.084511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.084499 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.103444 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.103409 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv"] Oct 30 18:46:21.107351 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.107330 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c"] Oct 30 18:46:21.107479 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.107443 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.110976 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.110958 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.130907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.128307 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:21.130907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.129156 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:21.130907 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.130411 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:21.138895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.138863 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:21.187658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.187601 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv"] Oct 30 18:46:21.189861 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.189826 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.192764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.192723 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.192994 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.192963 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.193080 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193043 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.193142 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193084 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193211 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193255 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193325 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193335 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193408 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqm2\" (UniqueName: \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193444 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7wf\" (UniqueName: \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.193519 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6fbv\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.194031 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194377 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.194275 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194995 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.194428 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.194995 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.194597 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.195387 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.195366 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.195495 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.194966 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.195650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.195633 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.196083 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.196051 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.196713 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.196696 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c"] Oct 30 18:46:21.205335 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.205310 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.207884 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.207862 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.230336 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.230300 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.230495 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.230366 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6fbv\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.297232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297146 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqm2\" (UniqueName: \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.297232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297189 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7wf\" (UniqueName: \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.297232 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297227 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.297452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297287 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.297452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297355 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.297452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.297408 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.298056 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.298035 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.298133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.298104 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.299748 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.299719 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.299842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.299760 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.300696 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.300679 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:21.335049 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.335012 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7wf\" (UniqueName: \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\") pod \"maas-api-revoke-sub-189cb936-qqj5c\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.336850 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.336822 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqm2\" (UniqueName: \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\") pod \"maas-api-revoke-sub-8806e007-wzkbv\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.422389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.422353 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:21.439735 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.439696 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:21.506513 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.506441 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd"] Oct 30 18:46:21.511071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.511020 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-a47385-zc7gd"] Oct 30 18:46:21.634123 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.634029 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:46:21.639781 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:21.639735 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c9e05a_cef4_4cba_a30b_e4efa8ec36ff.slice/crio-ff6fd2df2fc4b121b09b3e791cdd1745621620458082bc00bc4693118c3eb087 WatchSource:0}: Error finding container ff6fd2df2fc4b121b09b3e791cdd1745621620458082bc00bc4693118c3eb087: Status 404 returned error can't find the container with id ff6fd2df2fc4b121b09b3e791cdd1745621620458082bc00bc4693118c3eb087 Oct 30 18:46:21.642302 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.642270 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:46:21.642386 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.642336 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:46:21.642386 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.642364 6492 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:46:21.741964 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.741936 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv"] Oct 30 18:46:21.743732 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:21.743688 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d516a63_65f2_474a_97e2_7188bd728a8d.slice/crio-120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1 WatchSource:0}: Error finding container 120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1: Status 404 returned error can't find the container with id 120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1 Oct 30 18:46:21.781570 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.781529 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c"] Oct 30 18:46:21.955690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.955418 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:46:21.976749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.976483 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:21.976749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:21.976609 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:22.108358 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.108329 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\") pod \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " Oct 30 18:46:22.108795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.108406 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\" (UniqueName: \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\") pod \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\" (UID: \"f93ad3c0-ae40-40ad-bfe1-479dca147d8f\") " Oct 30 18:46:22.108795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.108769 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume" pod "f93ad3c0-ae40-40ad-bfe1-479dca147d8f" (UID: "f93ad3c0-ae40-40ad-bfe1-479dca147d8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:22.110665 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.110632 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b" pod "f93ad3c0-ae40-40ad-bfe1-479dca147d8f" (UID: "f93ad3c0-ae40-40ad-bfe1-479dca147d8f"). InnerVolumeSpecName "kube-api-access-dwl5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:22.126362 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.126337 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:46:22.211563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.210045 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\" (UniqueName: \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\") pod \"11ac06a6-6b24-404c-bc49-393723f87c1b\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " Oct 30 18:46:22.211563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.210115 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\") pod \"11ac06a6-6b24-404c-bc49-393723f87c1b\" (UID: \"11ac06a6-6b24-404c-bc49-393723f87c1b\") " Oct 30 18:46:22.211563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.210492 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:22.211563 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.210530 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwl5b\" (UniqueName: \"kubernetes.io/projected/f93ad3c0-ae40-40ad-bfe1-479dca147d8f-kube-api-access-dwl5b\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:22.213598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.213506 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume" pod "11ac06a6-6b24-404c-bc49-393723f87c1b" (UID: "11ac06a6-6b24-404c-bc49-393723f87c1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:22.237433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.236955 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg" pod "11ac06a6-6b24-404c-bc49-393723f87c1b" (UID: "11ac06a6-6b24-404c-bc49-393723f87c1b"). InnerVolumeSpecName "kube-api-access-9gdmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:22.316567 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.314324 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gdmg\" (UniqueName: \"kubernetes.io/projected/11ac06a6-6b24-404c-bc49-393723f87c1b-kube-api-access-9gdmg\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:22.316567 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.314365 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ac06a6-6b24-404c-bc49-393723f87c1b-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:22.367213 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.367180 6492 generic.go:356] "Generic (PLEG): container finished" podID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerID="59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8" exitCode=0 Oct 30 18:46:22.367400 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.367228 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" Oct 30 18:46:22.367400 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.367222 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" event={"ID":"f93ad3c0-ae40-40ad-bfe1-479dca147d8f","Type":"ContainerDied","Data":"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8"} Oct 30 18:46:22.367400 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.367306 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c" event={"ID":"f93ad3c0-ae40-40ad-bfe1-479dca147d8f","Type":"ContainerDied","Data":"d45310b8623277ef583d280a46e4b36a46060c33e2b2fd5139be4aa33fa03334"} Oct 30 18:46:22.367400 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.367345 6492 scope.go:122] "RemoveContainer" containerID="59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8" Oct 30 18:46:22.370213 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.369681 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" event={"ID":"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff","Type":"ContainerStarted","Data":"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc"} Oct 30 18:46:22.370213 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.369725 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" event={"ID":"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff","Type":"ContainerStarted","Data":"ff6fd2df2fc4b121b09b3e791cdd1745621620458082bc00bc4693118c3eb087"} Oct 30 18:46:22.371772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.371743 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" event={"ID":"0d516a63-65f2-474a-97e2-7188bd728a8d","Type":"ContainerStarted","Data":"648f7edabe5923b97849fd3a044525e0fbac53df1717ce7fa9150a66c25ddb30"} Oct 30 18:46:22.371870 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.371797 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" event={"ID":"0d516a63-65f2-474a-97e2-7188bd728a8d","Type":"ContainerStarted","Data":"120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1"} Oct 30 18:46:22.373837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.373767 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" event={"ID":"f28daf6b-fee5-4a60-bc85-88672e5681eb","Type":"ContainerStarted","Data":"c33663d03623bb4202e1d2fbb915a6089131bfa604ee14546fde22e0aec2566c"} Oct 30 18:46:22.373837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.373811 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" event={"ID":"f28daf6b-fee5-4a60-bc85-88672e5681eb","Type":"ContainerStarted","Data":"a238dedef9ff81822909c2af4d4f3cc43114cb390bf362d862b9938c3297821a"} Oct 30 18:46:22.375151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.375130 6492 generic.go:356] "Generic (PLEG): container finished" podID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerID="4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310" exitCode=0 Oct 30 18:46:22.375252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.375175 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" Oct 30 18:46:22.375252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.375182 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" event={"ID":"11ac06a6-6b24-404c-bc49-393723f87c1b","Type":"ContainerDied","Data":"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310"} Oct 30 18:46:22.375252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.375209 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h" event={"ID":"11ac06a6-6b24-404c-bc49-393723f87c1b","Type":"ContainerDied","Data":"a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e"} Oct 30 18:46:22.380697 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.380671 6492 scope.go:122] "RemoveContainer" containerID="59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8" Oct 30 18:46:22.383413 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:22.383387 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8\": container with ID starting with 59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8 not found: ID does not exist" containerID="59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8" Oct 30 18:46:22.383623 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.383585 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8"} err="failed to get container status \"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8\": rpc error: code = NotFound desc = could not find container \"59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8\": container with ID starting with 59db4219e52cbe41313b74656b6dcdfc7fb8e128c7ae61a27be3aac3695e0bb8 not found: ID does not exist" Oct 30 18:46:22.383623 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.383626 6492 scope.go:122] "RemoveContainer" containerID="4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310" Oct 30 18:46:22.385634 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:22.385614 6492 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ac06a6_6b24_404c_bc49_393723f87c1b.slice/crio-a8d786587ae8053871ed8ffa04322f1396aeb26f307540ed739b14d864ea2e3e\": RecentStats: unable to find data in memory cache]" Oct 30 18:46:22.397894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.397877 6492 scope.go:122] "RemoveContainer" containerID="4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310" Oct 30 18:46:22.398365 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:22.398303 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310\": container with ID starting with 4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310 not found: ID does not exist" containerID="4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310" Oct 30 18:46:22.398365 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.398336 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310"} err="failed to get container status \"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310\": rpc error: code = NotFound desc = could not find container \"4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310\": container with ID starting with 4b6b74312ff92b5629ae7dc50bb8df0dec52a0065b5d6a1b8d7276d715378310 not found: ID does not exist" Oct 30 18:46:22.446189 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.446160 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:46:22.468218 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.468146 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-a47385-5f469786c4-qwf5c"] Oct 30 18:46:22.540018 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.539959 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podStartSLOduration=2.539946712 podStartE2EDuration="2.539946712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:22.53679966 +0000 UTC m=+853.981190655" watchObservedRunningTime="2026-10-30 18:46:22.539946712 +0000 UTC m=+853.984337706" Oct 30 18:46:22.611759 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.611691 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" podStartSLOduration=1.611671244 podStartE2EDuration="1.611671244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:22.608913731 +0000 UTC m=+854.053304753" watchObservedRunningTime="2026-10-30 18:46:22.611671244 +0000 UTC m=+854.056062243" Oct 30 18:46:22.664671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.664606 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" podStartSLOduration=1.664588001 podStartE2EDuration="1.664588001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:22.66016149 +0000 UTC m=+854.104552486" watchObservedRunningTime="2026-10-30 18:46:22.664588001 +0000 UTC m=+854.108979000" Oct 30 18:46:22.665257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.665229 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8de70891-5dcc-45b1-858a-4ee86cf49bfc" path="/var/lib/kubelet/pods/8de70891-5dcc-45b1-858a-4ee86cf49bfc/volumes" Oct 30 18:46:22.665712 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.665693 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" path="/var/lib/kubelet/pods/f93ad3c0-ae40-40ad-bfe1-479dca147d8f/volumes" Oct 30 18:46:22.714634 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.714597 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:46:22.738450 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:22.738403 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-a47385-7495675595-cqb8h"] Oct 30 18:46:23.301998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.301910 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:23.306686 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.306655 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:23.306911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.306885 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:23.384051 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.383992 6492 generic.go:356] "Generic (PLEG): container finished" podID="f28daf6b-fee5-4a60-bc85-88672e5681eb" containerID="c33663d03623bb4202e1d2fbb915a6089131bfa604ee14546fde22e0aec2566c" exitCode=0 Oct 30 18:46:23.384242 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.384077 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" event={"ID":"f28daf6b-fee5-4a60-bc85-88672e5681eb","Type":"ContainerDied","Data":"c33663d03623bb4202e1d2fbb915a6089131bfa604ee14546fde22e0aec2566c"} Oct 30 18:46:23.387987 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.387959 6492 generic.go:356] "Generic (PLEG): container finished" podID="0d516a63-65f2-474a-97e2-7188bd728a8d" containerID="648f7edabe5923b97849fd3a044525e0fbac53df1717ce7fa9150a66c25ddb30" exitCode=0 Oct 30 18:46:23.388110 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.388071 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" event={"ID":"0d516a63-65f2-474a-97e2-7188bd728a8d","Type":"ContainerDied","Data":"648f7edabe5923b97849fd3a044525e0fbac53df1717ce7fa9150a66c25ddb30"} Oct 30 18:46:23.477793 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.477123 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" podUID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" containerName="main" containerID="cri-o://fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760" gracePeriod=45 Oct 30 18:46:23.842055 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.841978 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:46:23.864925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.864890 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.864984 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\" (UniqueName: \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865029 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\" (UniqueName: \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865070 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865121 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865157 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.865265 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865173 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\") pod \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\" (UID: \"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3\") " Oct 30 18:46:23.865652 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865476 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.865652 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865609 6492 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-model-cache\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.865815 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865662 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.865916 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.865894 6492 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-home\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.867385 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.867361 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.867727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.867691 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "kube-api-access-xhvjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:23.867892 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.867874 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs" pod "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" (UID: "4091c9d8-60d3-4b63-a9f7-673a3cf89bf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:23.967125 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.967080 6492 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tls-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.967125 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.967125 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhvjk\" (UniqueName: \"kubernetes.io/projected/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-kube-api-access-xhvjk\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.967406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.967141 6492 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-tmp-dir\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.967406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:23.967155 6492 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3-dshm\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.303435 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.302959 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:24.303435 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.303038 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:24.409628 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.396668 6492 generic.go:356] "Generic (PLEG): container finished" podID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" containerID="fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760" exitCode=0 Oct 30 18:46:24.409628 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.409602 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" event={"ID":"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3","Type":"ContainerDied","Data":"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760"} Oct 30 18:46:24.410474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.409671 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" event={"ID":"4091c9d8-60d3-4b63-a9f7-673a3cf89bf3","Type":"ContainerDied","Data":"af562f70121314ecb1eae322d4260ef49650383cb356af1dcede1b54a7788f5b"} Oct 30 18:46:24.410474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.409699 6492 scope.go:122] "RemoveContainer" containerID="fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760" Oct 30 18:46:24.410474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.409887 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj" Oct 30 18:46:24.437792 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.437765 6492 scope.go:122] "RemoveContainer" containerID="fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760" Oct 30 18:46:24.438163 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:24.438114 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760\": container with ID starting with fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760 not found: ID does not exist" containerID="fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760" Oct 30 18:46:24.438243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.438175 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760"} err="failed to get container status \"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760\": rpc error: code = NotFound desc = could not find container \"fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760\": container with ID starting with fcf298cefc9f18b0bf1efdc3dd63fc243804fe8484b3cb397b16aa2b731a4760 not found: ID does not exist" Oct 30 18:46:24.472835 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.472776 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:46:24.486770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.485961 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/simulator-w2-b2ef42-kserve-58dfdcdc66-hvddj"] Oct 30 18:46:24.537768 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.533680 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:46:24.546007 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.545639 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:46:24.663790 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.663435 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:24.666586 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.666525 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:24.669520 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.669491 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" path="/var/lib/kubelet/pods/11ac06a6-6b24-404c-bc49-393723f87c1b/volumes" Oct 30 18:46:24.669949 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.669932 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" path="/var/lib/kubelet/pods/4091c9d8-60d3-4b63-a9f7-673a3cf89bf3/volumes" Oct 30 18:46:24.778098 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778067 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\") pod \"f28daf6b-fee5-4a60-bc85-88672e5681eb\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " Oct 30 18:46:24.778278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778130 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\") pod \"f28daf6b-fee5-4a60-bc85-88672e5681eb\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " Oct 30 18:46:24.778278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778183 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\") pod \"0d516a63-65f2-474a-97e2-7188bd728a8d\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " Oct 30 18:46:24.778278 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778213 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\" (UniqueName: \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\") pod \"f28daf6b-fee5-4a60-bc85-88672e5681eb\" (UID: \"f28daf6b-fee5-4a60-bc85-88672e5681eb\") " Oct 30 18:46:24.778470 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778322 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\") pod \"0d516a63-65f2-474a-97e2-7188bd728a8d\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " Oct 30 18:46:24.778470 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778368 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\" (UniqueName: \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\") pod \"0d516a63-65f2-474a-97e2-7188bd728a8d\" (UID: \"0d516a63-65f2-474a-97e2-7188bd728a8d\") " Oct 30 18:46:24.778696 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778661 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca" pod "f28daf6b-fee5-4a60-bc85-88672e5681eb" (UID: "f28daf6b-fee5-4a60-bc85-88672e5681eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:24.778775 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.778737 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca" pod "0d516a63-65f2-474a-97e2-7188bd728a8d" (UID: "0d516a63-65f2-474a-97e2-7188bd728a8d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:24.779185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.779157 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.779185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.779183 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.781039 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.780998 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2" pod "0d516a63-65f2-474a-97e2-7188bd728a8d" (UID: "0d516a63-65f2-474a-97e2-7188bd728a8d"). InnerVolumeSpecName "kube-api-access-xkqm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:24.781157 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.781060 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca" pod "f28daf6b-fee5-4a60-bc85-88672e5681eb" (UID: "f28daf6b-fee5-4a60-bc85-88672e5681eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:24.781157 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.781105 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf" pod "f28daf6b-fee5-4a60-bc85-88672e5681eb" (UID: "f28daf6b-fee5-4a60-bc85-88672e5681eb"). InnerVolumeSpecName "kube-api-access-8p7wf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:24.781357 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.781336 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca" pod "0d516a63-65f2-474a-97e2-7188bd728a8d" (UID: "0d516a63-65f2-474a-97e2-7188bd728a8d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:24.880923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.880842 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f28daf6b-fee5-4a60-bc85-88672e5681eb-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.880923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.880880 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p7wf\" (UniqueName: \"kubernetes.io/projected/f28daf6b-fee5-4a60-bc85-88672e5681eb-kube-api-access-8p7wf\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.880923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.880895 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0d516a63-65f2-474a-97e2-7188bd728a8d-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.880923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:24.880912 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkqm2\" (UniqueName: \"kubernetes.io/projected/0d516a63-65f2-474a-97e2-7188bd728a8d-kube-api-access-xkqm2\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:25.316650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.312956 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:25.316650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.313034 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:25.415598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.415560 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" event={"ID":"0d516a63-65f2-474a-97e2-7188bd728a8d","Type":"ContainerDied","Data":"120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1"} Oct 30 18:46:25.415899 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.415873 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="120e12c4c65e3f4e7fadf6ebb0effd510293e4c11c0d65d3a7d401dab07651e1" Oct 30 18:46:25.416099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.416087 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8806e007-wzkbv" Oct 30 18:46:25.426220 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.426184 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" event={"ID":"f28daf6b-fee5-4a60-bc85-88672e5681eb","Type":"ContainerDied","Data":"a238dedef9ff81822909c2af4d4f3cc43114cb390bf362d862b9938c3297821a"} Oct 30 18:46:25.430638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.430613 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a238dedef9ff81822909c2af4d4f3cc43114cb390bf362d862b9938c3297821a" Oct 30 18:46:25.430809 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:25.426679 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-189cb936-qqj5c" Oct 30 18:46:26.305433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:26.304980 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:26.305433 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:26.305047 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:27.311003 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:27.310963 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:27.311630 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:27.311597 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:28.301923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.301886 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body= Oct 30 18:46:28.302217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.302186 6492 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" Oct 30 18:46:28.395434 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395403 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4"] Oct 30 18:46:28.395919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395906 6492 cpu_manager.go:414] "containerMap: removing container" podUID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" containerName="main" Oct 30 18:46:28.395971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395921 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" containerName="main" Oct 30 18:46:28.395971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395942 6492 cpu_manager.go:414] "containerMap: removing container" podUID="0d516a63-65f2-474a-97e2-7188bd728a8d" containerName="revoke-keys" Oct 30 18:46:28.395971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395947 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d516a63-65f2-474a-97e2-7188bd728a8d" containerName="revoke-keys" Oct 30 18:46:28.395971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395965 6492 cpu_manager.go:414] "containerMap: removing container" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" Oct 30 18:46:28.395971 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395969 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395987 6492 cpu_manager.go:414] "containerMap: removing container" podUID="f28daf6b-fee5-4a60-bc85-88672e5681eb" containerName="revoke-keys" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395991 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f28daf6b-fee5-4a60-bc85-88672e5681eb" containerName="revoke-keys" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.395999 6492 cpu_manager.go:414] "containerMap: removing container" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396004 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396068 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="f93ad3c0-ae40-40ad-bfe1-479dca147d8f" containerName="payload-processing" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396077 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="f28daf6b-fee5-4a60-bc85-88672e5681eb" containerName="revoke-keys" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396083 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="11ac06a6-6b24-404c-bc49-393723f87c1b" containerName="payload-pre-processing" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396089 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="4091c9d8-60d3-4b63-a9f7-673a3cf89bf3" containerName="main" Oct 30 18:46:28.396129 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.396096 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d516a63-65f2-474a-97e2-7188bd728a8d" containerName="revoke-keys" Oct 30 18:46:28.401077 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.401054 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.420821 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.420788 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:28.424172 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.424141 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j"] Oct 30 18:46:28.428349 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.428325 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:28.430678 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.430631 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:28.434615 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.434594 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4"] Oct 30 18:46:28.434995 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.434966 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.435758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.435693 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:28.445439 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.445407 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j"] Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556316 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67dl\" (UniqueName: \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556485 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9swpf\" (UniqueName: \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556531 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556668 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556715 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.557275 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.556740 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.657769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.657722 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.657946 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.657781 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.657946 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.657894 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67dl\" (UniqueName: \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.658078 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.657966 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9swpf\" (UniqueName: \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.658078 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.657999 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.658078 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.658062 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.658930 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.658633 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.659209 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.659181 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.661238 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.661194 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.662807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.662760 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.686723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.686691 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67dl\" (UniqueName: \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\") pod \"maas-api-revoke-sub-8b86648d-d6m6j\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:28.687105 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.687078 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9swpf\" (UniqueName: \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\") pod \"maas-api-revoke-sub-59766bc4-h9mj4\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.729332 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.729282 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:28.759015 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:28.758981 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:29.066474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.066433 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4"] Oct 30 18:46:29.072037 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:29.071997 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7070e575_6253_48fe_9325_8450c80fa6dd.slice/crio-89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4 WatchSource:0}: Error finding container 89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4: Status 404 returned error can't find the container with id 89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4 Oct 30 18:46:29.102977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.101724 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j"] Oct 30 18:46:29.104815 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:29.104787 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode530a0b4_122a_4223_bd2d_1677ba0754ec.slice/crio-3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4 WatchSource:0}: Error finding container 3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4: Status 404 returned error can't find the container with id 3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4 Oct 30 18:46:29.310143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.310115 6492 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:29.311312 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.311288 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:29.312154 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.312132 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:46:29.470304 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.470272 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" event={"ID":"e530a0b4-122a-4223-bd2d-1677ba0754ec","Type":"ContainerStarted","Data":"28714feb40b4bf0a1e7b3d7c9b5945d7538b74ac9578ea781add739c8475dff6"} Oct 30 18:46:29.470766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.470326 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" event={"ID":"e530a0b4-122a-4223-bd2d-1677ba0754ec","Type":"ContainerStarted","Data":"3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4"} Oct 30 18:46:29.472272 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.472243 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" event={"ID":"7070e575-6253-48fe-9325-8450c80fa6dd","Type":"ContainerStarted","Data":"bb4639ebbc50056e85d304bb372f4fb1ab54e7915a5057699517385ee3166d46"} Oct 30 18:46:29.472406 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.472295 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" event={"ID":"7070e575-6253-48fe-9325-8450c80fa6dd","Type":"ContainerStarted","Data":"89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4"} Oct 30 18:46:29.512306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.512246 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" podStartSLOduration=1.512231174 podStartE2EDuration="1.512231174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:29.510662097 +0000 UTC m=+860.955053124" watchObservedRunningTime="2026-10-30 18:46:29.512231174 +0000 UTC m=+860.956622205" Oct 30 18:46:29.534222 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.534185 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" containerID="cri-o://60ddc01f64011be3ea03069688a74b6aa2afb88ed36f12bfca8ef4424e695dde" gracePeriod=25 Oct 30 18:46:29.545998 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.545940 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" containerID="cri-o://3288d643743855344ab00cf392c52e30e6d06197538dd590248e8bc48f9688be" gracePeriod=25 Oct 30 18:46:29.558285 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:29.558235 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" podStartSLOduration=1.558221148 podStartE2EDuration="1.558221148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:29.556154748 +0000 UTC m=+861.000545764" watchObservedRunningTime="2026-10-30 18:46:29.558221148 +0000 UTC m=+861.002612142" Oct 30 18:46:30.482058 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:30.482027 6492 generic.go:356] "Generic (PLEG): container finished" podID="7070e575-6253-48fe-9325-8450c80fa6dd" containerID="bb4639ebbc50056e85d304bb372f4fb1ab54e7915a5057699517385ee3166d46" exitCode=0 Oct 30 18:46:30.482550 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:30.482119 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" event={"ID":"7070e575-6253-48fe-9325-8450c80fa6dd","Type":"ContainerDied","Data":"bb4639ebbc50056e85d304bb372f4fb1ab54e7915a5057699517385ee3166d46"} Oct 30 18:46:30.484105 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:30.484079 6492 generic.go:356] "Generic (PLEG): container finished" podID="e530a0b4-122a-4223-bd2d-1677ba0754ec" containerID="28714feb40b4bf0a1e7b3d7c9b5945d7538b74ac9578ea781add739c8475dff6" exitCode=0 Oct 30 18:46:30.484200 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:30.484156 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" event={"ID":"e530a0b4-122a-4223-bd2d-1677ba0754ec","Type":"ContainerDied","Data":"28714feb40b4bf0a1e7b3d7c9b5945d7538b74ac9578ea781add739c8475dff6"} Oct 30 18:46:31.680659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.680634 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:31.683861 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.683840 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:31.806655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806572 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\") pod \"7070e575-6253-48fe-9325-8450c80fa6dd\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " Oct 30 18:46:31.806655 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806645 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\" (UniqueName: \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\") pod \"e530a0b4-122a-4223-bd2d-1677ba0754ec\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " Oct 30 18:46:31.806883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806720 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\") pod \"e530a0b4-122a-4223-bd2d-1677ba0754ec\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " Oct 30 18:46:31.806883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806761 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\") pod \"e530a0b4-122a-4223-bd2d-1677ba0754ec\" (UID: \"e530a0b4-122a-4223-bd2d-1677ba0754ec\") " Oct 30 18:46:31.806883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806793 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\") pod \"7070e575-6253-48fe-9325-8450c80fa6dd\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " Oct 30 18:46:31.806883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.806876 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\" (UniqueName: \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\") pod \"7070e575-6253-48fe-9325-8450c80fa6dd\" (UID: \"7070e575-6253-48fe-9325-8450c80fa6dd\") " Oct 30 18:46:31.807665 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.807631 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca" pod "7070e575-6253-48fe-9325-8450c80fa6dd" (UID: "7070e575-6253-48fe-9325-8450c80fa6dd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:31.807832 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.807813 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca" pod "e530a0b4-122a-4223-bd2d-1677ba0754ec" (UID: "e530a0b4-122a-4223-bd2d-1677ba0754ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:31.811896 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.809263 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca" pod "7070e575-6253-48fe-9325-8450c80fa6dd" (UID: "7070e575-6253-48fe-9325-8450c80fa6dd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:31.811896 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.809595 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf" pod "7070e575-6253-48fe-9325-8450c80fa6dd" (UID: "7070e575-6253-48fe-9325-8450c80fa6dd"). InnerVolumeSpecName "kube-api-access-9swpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:31.811896 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.812204 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca" pod "e530a0b4-122a-4223-bd2d-1677ba0754ec" (UID: "e530a0b4-122a-4223-bd2d-1677ba0754ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:31.811896 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.813657 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl" pod "e530a0b4-122a-4223-bd2d-1677ba0754ec" (UID: "e530a0b4-122a-4223-bd2d-1677ba0754ec"). InnerVolumeSpecName "kube-api-access-g67dl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:31.908175 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908134 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g67dl\" (UniqueName: \"kubernetes.io/projected/e530a0b4-122a-4223-bd2d-1677ba0754ec-kube-api-access-g67dl\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.908175 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908169 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.908175 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908180 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e530a0b4-122a-4223-bd2d-1677ba0754ec-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.908404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908190 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.908404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908198 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9swpf\" (UniqueName: \"kubernetes.io/projected/7070e575-6253-48fe-9325-8450c80fa6dd-kube-api-access-9swpf\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.908404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.908206 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7070e575-6253-48fe-9325-8450c80fa6dd-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:31.976217 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.976177 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:31.976384 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:31.976187 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:32.311250 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.311211 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:46:32.311604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.311486 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" containerID="cri-o://c80b8f5c2541f6972c73c5fab5f2e891c22f0467d56fb3a0f44cf7df573701b8" gracePeriod=30 Oct 30 18:46:32.506053 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.506017 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" Oct 30 18:46:32.506248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.506047 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59766bc4-h9mj4" event={"ID":"7070e575-6253-48fe-9325-8450c80fa6dd","Type":"ContainerDied","Data":"89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4"} Oct 30 18:46:32.506248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.506097 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89516036ff870046ad30f6a022fe19fb8b7e38e50411863cd8d381f941b8fee4" Oct 30 18:46:32.507680 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.507661 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" Oct 30 18:46:32.507831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.507662 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8b86648d-d6m6j" event={"ID":"e530a0b4-122a-4223-bd2d-1677ba0754ec","Type":"ContainerDied","Data":"3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4"} Oct 30 18:46:32.507831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.507790 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f4e6ee6fcdf157c998f3137aa4b53b657c1dd2b2aebbb1aa2100dbe415ae2f4" Oct 30 18:46:32.983690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.983654 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:32.983690 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:32.983684 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:33.796014 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:33.795965 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" podUID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" containerName="main" containerID="cri-o://bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1" gracePeriod=45 Oct 30 18:46:34.070567 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.070524 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:46:34.137042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137008 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.137294 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137273 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.137482 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137445 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\" (UniqueName: \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.137613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137519 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.137691 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137630 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.137745 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137686 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\" (UniqueName: \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\") pod \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\" (UID: \"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0\") " Oct 30 18:46:34.139795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.137360 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.139795 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.139519 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.139989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.139705 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.141409 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.140731 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "kube-api-access-j565b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:34.141798 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.141772 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.141940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.141924 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs" pod "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" (UID: "cf957d7c-0d1d-4b6c-92a1-be785a4c16a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:34.238571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238526 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j565b\" (UniqueName: \"kubernetes.io/projected/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-kube-api-access-j565b\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.238571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238571 6492 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-model-cache\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.238758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238583 6492 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-dshm\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.238758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238595 6492 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tls-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.238758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238605 6492 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-home\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.238758 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.238616 6492 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0-tmp-dir\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.526420 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.526380 6492 generic.go:356] "Generic (PLEG): container finished" podID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" containerID="bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1" exitCode=0 Oct 30 18:46:34.526659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.526430 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" Oct 30 18:46:34.526659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.526429 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" event={"ID":"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0","Type":"ContainerDied","Data":"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1"} Oct 30 18:46:34.526659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.526489 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh" event={"ID":"cf957d7c-0d1d-4b6c-92a1-be785a4c16a0","Type":"ContainerDied","Data":"bd1e1a20fd5b48a4e17c1944558f00b989a5a6bba8321518f9c093f9be56d958"} Oct 30 18:46:34.526659 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.526532 6492 scope.go:122] "RemoveContainer" containerID="bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1" Oct 30 18:46:34.548092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.548066 6492 scope.go:122] "RemoveContainer" containerID="bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1" Oct 30 18:46:34.548517 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:34.548456 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1\": container with ID starting with bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1 not found: ID does not exist" containerID="bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1" Oct 30 18:46:34.548517 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.548491 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1"} err="failed to get container status \"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1\": rpc error: code = NotFound desc = could not find container \"bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1\": container with ID starting with bad64b4c8eb69fcb88fde91a38a82c425187923332bcef5c20389dbf123671d1 not found: ID does not exist" Oct 30 18:46:34.586934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.586897 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:46:34.589739 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.589710 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/simulator-w0-f6f7d5-kserve-cc7f6d87f-cc4nh"] Oct 30 18:46:34.667272 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:34.667231 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" path="/var/lib/kubelet/pods/cf957d7c-0d1d-4b6c-92a1-be785a4c16a0/volumes" Oct 30 18:46:35.237852 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:35.237819 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj"] Oct 30 18:46:35.249459 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:35.249425 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-b2ef42-2f2lj"] Oct 30 18:46:36.664964 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:36.664931 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27987f12-1a35-44ec-aa68-d59a5b772e65" path="/var/lib/kubelet/pods/27987f12-1a35-44ec-aa68-d59a5b772e65/volumes" Oct 30 18:46:37.553511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.553476 6492 generic.go:356] "Generic (PLEG): container finished" podID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerID="c80b8f5c2541f6972c73c5fab5f2e891c22f0467d56fb3a0f44cf7df573701b8" exitCode=0 Oct 30 18:46:37.553710 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.553551 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" event={"ID":"08925c8d-f8a6-4b35-bc06-3329113b654a","Type":"ContainerDied","Data":"c80b8f5c2541f6972c73c5fab5f2e891c22f0467d56fb3a0f44cf7df573701b8"} Oct 30 18:46:37.596191 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.596133 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:46:37.671926 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.671886 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.671947 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672025 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672084 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672143 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672175 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672178 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672236 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672282 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672346 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\") pod \"08925c8d-f8a6-4b35-bc06-3329113b654a\" (UID: \"08925c8d-f8a6-4b35-bc06-3329113b654a\") " Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672588 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672667 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672788 6492 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672807 6492 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08925c8d-f8a6-4b35-bc06-3329113b654a-istiod-ca-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672821 6492 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-workload-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672839 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.672919 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.672898 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.674728 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.674704 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.674865 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.674783 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "kube-api-access-dhd4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:37.675191 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.675175 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:37.675268 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.675208 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token" pod "08925c8d-f8a6-4b35-bc06-3329113b654a" (UID: "08925c8d-f8a6-4b35-bc06-3329113b654a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:37.773307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773267 6492 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-envoy\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.773307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773301 6492 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-token\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.773307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773310 6492 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-podinfo\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.773590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773318 6492 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-credential-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.773590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773328 6492 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08925c8d-f8a6-4b35-bc06-3329113b654a-istio-data\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.773590 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:37.773336 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhd4t\" (UniqueName: \"kubernetes.io/projected/08925c8d-f8a6-4b35-bc06-3329113b654a-kube-api-access-dhd4t\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:38.563847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.563803 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" event={"ID":"08925c8d-f8a6-4b35-bc06-3329113b654a","Type":"ContainerDied","Data":"2eef5eded4fb7d67532e34ee9a17789671fa881ce7a0365c40d739041d5d7374"} Oct 30 18:46:38.563847 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.563836 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm" Oct 30 18:46:38.564107 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.563875 6492 scope.go:122] "RemoveContainer" containerID="c80b8f5c2541f6972c73c5fab5f2e891c22f0467d56fb3a0f44cf7df573701b8" Oct 30 18:46:38.636706 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.636665 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:46:38.644033 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.643998 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-4dkmm"] Oct 30 18:46:38.673346 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:38.673302 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" path="/var/lib/kubelet/pods/08925c8d-f8a6-4b35-bc06-3329113b654a/volumes" Oct 30 18:46:39.601729 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:39.601690 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:46:39.984959 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:39.984932 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:46:39.999061 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:39.999031 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\" (UniqueName: \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\") pod \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " Oct 30 18:46:39.999255 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:39.999069 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\") pod \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\" (UID: \"51c92a0a-fa51-49a1-b6a7-738d74001a9f\") " Oct 30 18:46:39.999587 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:39.999558 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume" pod "51c92a0a-fa51-49a1-b6a7-738d74001a9f" (UID: "51c92a0a-fa51-49a1-b6a7-738d74001a9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:40.001162 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.001140 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf" pod "51c92a0a-fa51-49a1-b6a7-738d74001a9f" (UID: "51c92a0a-fa51-49a1-b6a7-738d74001a9f"). InnerVolumeSpecName "kube-api-access-sfjhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:40.018041 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.018013 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:46:40.099704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.099654 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\") pod \"fa26e794-20f3-46cc-985e-6e97ed4c5599\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " Oct 30 18:46:40.099704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.099733 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\" (UniqueName: \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\") pod \"fa26e794-20f3-46cc-985e-6e97ed4c5599\" (UID: \"fa26e794-20f3-46cc-985e-6e97ed4c5599\") " Oct 30 18:46:40.100190 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.100151 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume" pod "fa26e794-20f3-46cc-985e-6e97ed4c5599" (UID: "fa26e794-20f3-46cc-985e-6e97ed4c5599"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:40.100369 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.100282 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfjhf\" (UniqueName: \"kubernetes.io/projected/51c92a0a-fa51-49a1-b6a7-738d74001a9f-kube-api-access-sfjhf\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:40.100369 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.100305 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c92a0a-fa51-49a1-b6a7-738d74001a9f-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:40.100369 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.100320 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa26e794-20f3-46cc-985e-6e97ed4c5599-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:40.103337 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.102335 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b" pod "fa26e794-20f3-46cc-985e-6e97ed4c5599" (UID: "fa26e794-20f3-46cc-985e-6e97ed4c5599"). InnerVolumeSpecName "kube-api-access-tc46b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:40.200974 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.200939 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc46b\" (UniqueName: \"kubernetes.io/projected/fa26e794-20f3-46cc-985e-6e97ed4c5599-kube-api-access-tc46b\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:40.224819 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.224785 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:46:40.296533 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.296499 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh"] Oct 30 18:46:40.297065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297036 6492 cpu_manager.go:414] "containerMap: removing container" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" Oct 30 18:46:40.297065 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297063 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297092 6492 cpu_manager.go:414] "containerMap: removing container" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297101 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297132 6492 cpu_manager.go:414] "containerMap: removing container" podUID="7070e575-6253-48fe-9325-8450c80fa6dd" containerName="revoke-keys" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297141 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7070e575-6253-48fe-9325-8450c80fa6dd" containerName="revoke-keys" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297152 6492 cpu_manager.go:414] "containerMap: removing container" podUID="e530a0b4-122a-4223-bd2d-1677ba0754ec" containerName="revoke-keys" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297159 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e530a0b4-122a-4223-bd2d-1677ba0754ec" containerName="revoke-keys" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297173 6492 cpu_manager.go:414] "containerMap: removing container" podUID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" containerName="main" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297177 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" containerName="main" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297185 6492 cpu_manager.go:414] "containerMap: removing container" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297189 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297252 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="08925c8d-f8a6-4b35-bc06-3329113b654a" containerName="istio-proxy" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297263 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerName="payload-processing" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297270 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerName="payload-pre-processing" Oct 30 18:46:40.297273 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297278 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="7070e575-6253-48fe-9325-8450c80fa6dd" containerName="revoke-keys" Oct 30 18:46:40.297829 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297283 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="e530a0b4-122a-4223-bd2d-1677ba0754ec" containerName="revoke-keys" Oct 30 18:46:40.297829 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.297290 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf957d7c-0d1d-4b6c-92a1-be785a4c16a0" containerName="main" Oct 30 18:46:40.301183 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.301160 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.345608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.343323 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:40.345608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.343450 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:40.345608 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.343988 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh"] Oct 30 18:46:40.350351 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.346579 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:40.350351 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.346712 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:40.403881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.403275 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.405872 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.405533 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvlqd\" (UniqueName: \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.405872 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.405599 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.507913 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.507824 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.507913 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.507875 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvlqd\" (UniqueName: \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.507913 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.507903 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.508558 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.508521 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.510483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.510453 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.540938 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.540898 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvlqd\" (UniqueName: \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\") pod \"maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.588170 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.588117 6492 generic.go:356] "Generic (PLEG): container finished" podID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" containerID="dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18" exitCode=0 Oct 30 18:46:40.588170 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.588163 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" Oct 30 18:46:40.588387 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.588198 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" event={"ID":"51c92a0a-fa51-49a1-b6a7-738d74001a9f","Type":"ContainerDied","Data":"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18"} Oct 30 18:46:40.588387 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.588250 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj" event={"ID":"51c92a0a-fa51-49a1-b6a7-738d74001a9f","Type":"ContainerDied","Data":"03b240db0ce46603c06c339d6042d6e161b7832f9dba40bf4d5548cde7854b2d"} Oct 30 18:46:40.588387 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.588272 6492 scope.go:122] "RemoveContainer" containerID="dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18" Oct 30 18:46:40.590141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.590116 6492 generic.go:356] "Generic (PLEG): container finished" podID="fa26e794-20f3-46cc-985e-6e97ed4c5599" containerID="640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd" exitCode=0 Oct 30 18:46:40.590247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.590162 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" event={"ID":"fa26e794-20f3-46cc-985e-6e97ed4c5599","Type":"ContainerDied","Data":"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd"} Oct 30 18:46:40.590247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.590177 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" Oct 30 18:46:40.590247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.590202 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm" event={"ID":"fa26e794-20f3-46cc-985e-6e97ed4c5599","Type":"ContainerDied","Data":"d32d308837cd668fc59ad1856be92344a83d25f531f204eae26b991610783713"} Oct 30 18:46:40.600937 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.600915 6492 scope.go:122] "RemoveContainer" containerID="dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18" Oct 30 18:46:40.601280 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:40.601260 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18\": container with ID starting with dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18 not found: ID does not exist" containerID="dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18" Oct 30 18:46:40.601430 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.601292 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18"} err="failed to get container status \"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18\": rpc error: code = NotFound desc = could not find container \"dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18\": container with ID starting with dcc6fbf64b32aca70e739acd6ec4519ebf62ad788574bcd8518b352b6806af18 not found: ID does not exist" Oct 30 18:46:40.601430 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.601310 6492 scope.go:122] "RemoveContainer" containerID="640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd" Oct 30 18:46:40.611727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.611693 6492 scope.go:122] "RemoveContainer" containerID="640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd" Oct 30 18:46:40.612095 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:40.612066 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd\": container with ID starting with 640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd not found: ID does not exist" containerID="640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd" Oct 30 18:46:40.612205 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.612105 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd"} err="failed to get container status \"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd\": rpc error: code = NotFound desc = could not find container \"640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd\": container with ID starting with 640f680efc7cf4427502d35acfe755ff03ba4df37254eec154dc72c1f1fef2cd not found: ID does not exist" Oct 30 18:46:40.617467 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.617440 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:40.648891 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.648851 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:46:40.653607 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.653578 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-b2ef42-79fb677864-pscfj"] Oct 30 18:46:40.666910 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.666870 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51c92a0a-fa51-49a1-b6a7-738d74001a9f" path="/var/lib/kubelet/pods/51c92a0a-fa51-49a1-b6a7-738d74001a9f/volumes" Oct 30 18:46:40.693994 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.693930 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:46:40.697941 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.696782 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-b2ef42-6dfc59cf79-v5scm"] Oct 30 18:46:40.993178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:40.990619 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh"] Oct 30 18:46:41.600670 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:41.600564 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" event={"ID":"4659b622-5e9f-4922-9327-b7d95380b972","Type":"ContainerStarted","Data":"7d5eec4de9d98816073bf733be0b09f7ba7c5057eff1b25b59aa6270b7662a41"} Oct 30 18:46:41.600670 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:41.600662 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" event={"ID":"4659b622-5e9f-4922-9327-b7d95380b972","Type":"ContainerStarted","Data":"87ac1ae092583699eb982a213219fd4b20f825616b98634343f2ac6dbc8655a4"} Oct 30 18:46:41.652917 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:41.652851 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" podStartSLOduration=1.652838215 podStartE2EDuration="1.652838215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:41.651987773 +0000 UTC m=+873.096378766" watchObservedRunningTime="2026-10-30 18:46:41.652838215 +0000 UTC m=+873.097229245" Oct 30 18:46:41.683325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:41.683255 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf"] Oct 30 18:46:41.689740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:41.689705 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f6f7d5-nfhvf"] Oct 30 18:46:42.610879 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.610841 6492 generic.go:356] "Generic (PLEG): container finished" podID="4659b622-5e9f-4922-9327-b7d95380b972" containerID="7d5eec4de9d98816073bf733be0b09f7ba7c5057eff1b25b59aa6270b7662a41" exitCode=0 Oct 30 18:46:42.611305 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.610939 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" event={"ID":"4659b622-5e9f-4922-9327-b7d95380b972","Type":"ContainerDied","Data":"7d5eec4de9d98816073bf733be0b09f7ba7c5057eff1b25b59aa6270b7662a41"} Oct 30 18:46:42.664822 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.664786 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e9ae676-4a3f-40f5-8551-8e2316b3a533" path="/var/lib/kubelet/pods/0e9ae676-4a3f-40f5-8551-8e2316b3a533/volumes" Oct 30 18:46:42.665191 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.665177 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa26e794-20f3-46cc-985e-6e97ed4c5599" path="/var/lib/kubelet/pods/fa26e794-20f3-46cc-985e-6e97ed4c5599/volumes" Oct 30 18:46:42.983611 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.983575 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:42.983809 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:42.983581 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:43.199069 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.199026 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq"] Oct 30 18:46:43.203250 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.203225 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.238925 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.238849 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq"] Oct 30 18:46:43.254646 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.254606 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb4wn\" (UniqueName: \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.254837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.254755 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.254940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.254913 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.356130 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.356090 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.356316 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.356270 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.356399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.356382 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb4wn\" (UniqueName: \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.356848 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.356819 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.358527 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.358485 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.380412 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.380389 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb4wn\" (UniqueName: \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\") pod \"maas-api-revoke-sub-3a794a61-4j9mq\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.518888 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.518792 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:43.788955 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.788926 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:43.829956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.829925 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq"] Oct 30 18:46:43.831183 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:43.831144 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077cdcb0_6ac6_4f73_b65b_95ba34759d6d.slice/crio-896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c WatchSource:0}: Error finding container 896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c: Status 404 returned error can't find the container with id 896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c Oct 30 18:46:43.863031 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.863005 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\") pod \"4659b622-5e9f-4922-9327-b7d95380b972\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " Oct 30 18:46:43.863150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.863064 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\") pod \"4659b622-5e9f-4922-9327-b7d95380b972\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " Oct 30 18:46:43.863150 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.863141 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\" (UniqueName: \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\") pod \"4659b622-5e9f-4922-9327-b7d95380b972\" (UID: \"4659b622-5e9f-4922-9327-b7d95380b972\") " Oct 30 18:46:43.863523 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.863491 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca" pod "4659b622-5e9f-4922-9327-b7d95380b972" (UID: "4659b622-5e9f-4922-9327-b7d95380b972"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:43.865328 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.865300 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd" pod "4659b622-5e9f-4922-9327-b7d95380b972" (UID: "4659b622-5e9f-4922-9327-b7d95380b972"). InnerVolumeSpecName "kube-api-access-bvlqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:43.865419 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.865306 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca" pod "4659b622-5e9f-4922-9327-b7d95380b972" (UID: "4659b622-5e9f-4922-9327-b7d95380b972"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:43.964613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.964525 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:43.964613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.964570 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4659b622-5e9f-4922-9327-b7d95380b972-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:43.964613 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:43.964584 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvlqd\" (UniqueName: \"kubernetes.io/projected/4659b622-5e9f-4922-9327-b7d95380b972-kube-api-access-bvlqd\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:44.472182 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.472147 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:46:44.472419 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.472391 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" containerID="cri-o://a2635bd8cdc53b0bd495ea7bdc005543d11c1a5030b7b28021583695b5874b09" gracePeriod=30 Oct 30 18:46:44.557857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.557813 6492 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:46:44.558227 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.558188 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:46:44.633099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.633069 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" Oct 30 18:46:44.633343 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.633119 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh" event={"ID":"4659b622-5e9f-4922-9327-b7d95380b972","Type":"ContainerDied","Data":"87ac1ae092583699eb982a213219fd4b20f825616b98634343f2ac6dbc8655a4"} Oct 30 18:46:44.633446 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.633429 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87ac1ae092583699eb982a213219fd4b20f825616b98634343f2ac6dbc8655a4" Oct 30 18:46:44.635170 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.635123 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" event={"ID":"077cdcb0-6ac6-4f73-b65b-95ba34759d6d","Type":"ContainerStarted","Data":"5a86591ba57be423aad5a201eb7b453590ea23b0aa2128b57ced3bf1593829b6"} Oct 30 18:46:44.635170 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.635169 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" event={"ID":"077cdcb0-6ac6-4f73-b65b-95ba34759d6d","Type":"ContainerStarted","Data":"896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c"} Oct 30 18:46:44.686094 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:44.686029 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" podStartSLOduration=1.686010196 podStartE2EDuration="1.686010196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:44.684388546 +0000 UTC m=+876.128779541" watchObservedRunningTime="2026-10-30 18:46:44.686010196 +0000 UTC m=+876.130401191" Oct 30 18:46:45.225948 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:45.225897 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" containerID="cri-o://af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0" gracePeriod=25 Oct 30 18:46:45.644341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:45.644255 6492 generic.go:356] "Generic (PLEG): container finished" podID="077cdcb0-6ac6-4f73-b65b-95ba34759d6d" containerID="5a86591ba57be423aad5a201eb7b453590ea23b0aa2128b57ced3bf1593829b6" exitCode=0 Oct 30 18:46:45.644492 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:45.644333 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" event={"ID":"077cdcb0-6ac6-4f73-b65b-95ba34759d6d","Type":"ContainerDied","Data":"5a86591ba57be423aad5a201eb7b453590ea23b0aa2128b57ced3bf1593829b6"} Oct 30 18:46:46.788790 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.788757 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:46.899138 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.899101 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\" (UniqueName: \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\") pod \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " Oct 30 18:46:46.899307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.899214 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\") pod \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " Oct 30 18:46:46.899307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.899281 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\") pod \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\" (UID: \"077cdcb0-6ac6-4f73-b65b-95ba34759d6d\") " Oct 30 18:46:46.899667 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.899641 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca" pod "077cdcb0-6ac6-4f73-b65b-95ba34759d6d" (UID: "077cdcb0-6ac6-4f73-b65b-95ba34759d6d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:46.901298 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.901272 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn" pod "077cdcb0-6ac6-4f73-b65b-95ba34759d6d" (UID: "077cdcb0-6ac6-4f73-b65b-95ba34759d6d"). InnerVolumeSpecName "kube-api-access-zb4wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:46.901371 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:46.901337 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca" pod "077cdcb0-6ac6-4f73-b65b-95ba34759d6d" (UID: "077cdcb0-6ac6-4f73-b65b-95ba34759d6d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:46.976569 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:46.976500 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:46:47.000930 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.000901 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb4wn\" (UniqueName: \"kubernetes.io/projected/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-kube-api-access-zb4wn\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:47.000930 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.000929 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:47.000930 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.000939 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/077cdcb0-6ac6-4f73-b65b-95ba34759d6d-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:47.014934 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.014888 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:47.660245 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.660214 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" Oct 30 18:46:47.660480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.660212 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a794a61-4j9mq" event={"ID":"077cdcb0-6ac6-4f73-b65b-95ba34759d6d","Type":"ContainerDied","Data":"896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c"} Oct 30 18:46:47.660480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:47.660342 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="896ae1e378f1e833d10d310d8fc46a0146e406cb74a7d40004e6e0fa080b876c" Oct 30 18:46:49.697856 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.697812 6492 generic.go:356] "Generic (PLEG): container finished" podID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerID="60ddc01f64011be3ea03069688a74b6aa2afb88ed36f12bfca8ef4424e695dde" exitCode=0 Oct 30 18:46:49.698568 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.697917 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" event={"ID":"ed9a251c-c576-4e6c-87fa-6d295f0c7833","Type":"ContainerDied","Data":"60ddc01f64011be3ea03069688a74b6aa2afb88ed36f12bfca8ef4424e695dde"} Oct 30 18:46:49.700452 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.700419 6492 generic.go:356] "Generic (PLEG): container finished" podID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerID="a2635bd8cdc53b0bd495ea7bdc005543d11c1a5030b7b28021583695b5874b09" exitCode=0 Oct 30 18:46:49.700648 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.700625 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" event={"ID":"4c0902b1-64b3-4fae-a058-8adb0c1fc111","Type":"ContainerDied","Data":"a2635bd8cdc53b0bd495ea7bdc005543d11c1a5030b7b28021583695b5874b09"} Oct 30 18:46:49.703185 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.703161 6492 generic.go:356] "Generic (PLEG): container finished" podID="434d5008-fd97-47ad-81cf-64385b25936d" containerID="3288d643743855344ab00cf392c52e30e6d06197538dd590248e8bc48f9688be" exitCode=0 Oct 30 18:46:49.703325 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.703308 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" event={"ID":"434d5008-fd97-47ad-81cf-64385b25936d","Type":"ContainerDied","Data":"3288d643743855344ab00cf392c52e30e6d06197538dd590248e8bc48f9688be"} Oct 30 18:46:49.829496 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.829464 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:46:49.884002 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.883974 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:46:49.887475 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.887448 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:46:49.931831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.931799 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.931857 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.931888 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.931939 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.931969 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932010 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932000 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\" (UniqueName: \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\") pod \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932031 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932063 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\") pod \"434d5008-fd97-47ad-81cf-64385b25936d\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932092 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932126 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932156 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\" (UniqueName: \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\") pod \"434d5008-fd97-47ad-81cf-64385b25936d\" (UID: \"434d5008-fd97-47ad-81cf-64385b25936d\") " Oct 30 18:46:49.932248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932205 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\") pod \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\" (UID: \"ed9a251c-c576-4e6c-87fa-6d295f0c7833\") " Oct 30 18:46:49.932521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932268 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\") pod \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\" (UID: \"4c0902b1-64b3-4fae-a058-8adb0c1fc111\") " Oct 30 18:46:49.932521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932316 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.932521 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932440 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.932778 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.932756 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume" pod "434d5008-fd97-47ad-81cf-64385b25936d" (UID: "434d5008-fd97-47ad-81cf-64385b25936d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:49.933769 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.933740 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume" pod "ed9a251c-c576-4e6c-87fa-6d295f0c7833" (UID: "ed9a251c-c576-4e6c-87fa-6d295f0c7833"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:49.933964 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.933943 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:49.934111 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934088 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934601 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934687 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed9a251c-c576-4e6c-87fa-6d295f0c7833-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934709 6492 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istiod-ca-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934726 6492 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-data\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934739 6492 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934754 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d5008-fd97-47ad-81cf-64385b25936d-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.935895 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.934768 6492 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-workload-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.936388 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.936367 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.936482 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.936402 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w" pod "ed9a251c-c576-4e6c-87fa-6d295f0c7833" (UID: "ed9a251c-c576-4e6c-87fa-6d295f0c7833"). InnerVolumeSpecName "kube-api-access-6qv8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.936705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.936626 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "kube-api-access-xvjms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.936705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.936652 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z" pod "434d5008-fd97-47ad-81cf-64385b25936d" (UID: "434d5008-fd97-47ad-81cf-64385b25936d"). InnerVolumeSpecName "kube-api-access-sbs2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.936705 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.936670 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.937819 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:49.937802 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo" pod "4c0902b1-64b3-4fae-a058-8adb0c1fc111" (UID: "4c0902b1-64b3-4fae-a058-8adb0c1fc111"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:50.035954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035922 6492 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-credential-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.035954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035949 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qv8w\" (UniqueName: \"kubernetes.io/projected/ed9a251c-c576-4e6c-87fa-6d295f0c7833-kube-api-access-6qv8w\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.035954 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035961 6492 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-token\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.036177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035969 6492 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-podinfo\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.036177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035978 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbs2z\" (UniqueName: \"kubernetes.io/projected/434d5008-fd97-47ad-81cf-64385b25936d-kube-api-access-sbs2z\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.036177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035986 6492 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c0902b1-64b3-4fae-a058-8adb0c1fc111-istio-envoy\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.036177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.035996 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvjms\" (UniqueName: \"kubernetes.io/projected/4c0902b1-64b3-4fae-a058-8adb0c1fc111-kube-api-access-xvjms\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.712764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.712733 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" Oct 30 18:46:50.712764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.712745 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf" event={"ID":"ed9a251c-c576-4e6c-87fa-6d295f0c7833","Type":"ContainerDied","Data":"537c9a257b85c4375d7835a2a88be43d2a78d6c292ddf7d8878d948203031969"} Oct 30 18:46:50.713307 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.712818 6492 scope.go:122] "RemoveContainer" containerID="60ddc01f64011be3ea03069688a74b6aa2afb88ed36f12bfca8ef4424e695dde" Oct 30 18:46:50.715801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.715171 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" Oct 30 18:46:50.715801 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.715269 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q" event={"ID":"4c0902b1-64b3-4fae-a058-8adb0c1fc111","Type":"ContainerDied","Data":"5ea0e3def979ed2cc94765839607c419b1bc5953b8d4fb2348b3d5daaffe661a"} Oct 30 18:46:50.717417 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.717395 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" event={"ID":"434d5008-fd97-47ad-81cf-64385b25936d","Type":"ContainerDied","Data":"edd65c667e10053fa34346ba842b4a39c52e1c32b17bafeb5644854fcf4b6452"} Oct 30 18:46:50.717505 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.717479 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz" Oct 30 18:46:50.725426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.725405 6492 scope.go:122] "RemoveContainer" containerID="a2635bd8cdc53b0bd495ea7bdc005543d11c1a5030b7b28021583695b5874b09" Oct 30 18:46:50.736571 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.736553 6492 scope.go:122] "RemoveContainer" containerID="3288d643743855344ab00cf392c52e30e6d06197538dd590248e8bc48f9688be" Oct 30 18:46:50.773483 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.773432 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:46:50.785789 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.785765 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f6f7d5-97c74567d-m5qpf"] Oct 30 18:46:50.788598 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.788577 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:46:50.822427 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.822236 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:46:50.826835 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.826235 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-x9m2q"] Oct 30 18:46:50.867575 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.867529 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:46:50.873405 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:50.873378 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f6f7d5-569c9c7c5f-x7ncz"] Oct 30 18:46:51.021610 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:51.021494 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:46:51.021825 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:51.021797 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" containerID="cri-o://2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58" gracePeriod=30 Oct 30 18:46:52.428503 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.428465 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd"] Oct 30 18:46:52.429112 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429090 6492 cpu_manager.go:414] "containerMap: removing container" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" Oct 30 18:46:52.429160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429116 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" Oct 30 18:46:52.429160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429142 6492 cpu_manager.go:414] "containerMap: removing container" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" Oct 30 18:46:52.429160 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429150 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429198 6492 cpu_manager.go:414] "containerMap: removing container" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429207 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429227 6492 cpu_manager.go:414] "containerMap: removing container" podUID="4659b622-5e9f-4922-9327-b7d95380b972" containerName="revoke-keys" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429234 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4659b622-5e9f-4922-9327-b7d95380b972" containerName="revoke-keys" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429248 6492 cpu_manager.go:414] "containerMap: removing container" podUID="077cdcb0-6ac6-4f73-b65b-95ba34759d6d" containerName="revoke-keys" Oct 30 18:46:52.429264 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429256 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="077cdcb0-6ac6-4f73-b65b-95ba34759d6d" containerName="revoke-keys" Oct 30 18:46:52.429469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429350 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" containerName="payload-processing" Oct 30 18:46:52.429469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429393 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" containerName="istio-proxy" Oct 30 18:46:52.429469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429406 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="4659b622-5e9f-4922-9327-b7d95380b972" containerName="revoke-keys" Oct 30 18:46:52.429469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429415 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="077cdcb0-6ac6-4f73-b65b-95ba34759d6d" containerName="revoke-keys" Oct 30 18:46:52.429469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.429427 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="434d5008-fd97-47ad-81cf-64385b25936d" containerName="payload-pre-processing" Oct 30 18:46:52.434703 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.434680 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.458143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.458110 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:52.459023 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.458990 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf"] Oct 30 18:46:52.467145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.466609 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:52.467145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.466638 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:46:52.467145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.466707 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:46:52.467145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.466711 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd"] Oct 30 18:46:52.467145 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.466797 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.483766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.483738 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf"] Oct 30 18:46:52.564004 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.563952 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4whd9\" (UniqueName: \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.564176 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.564015 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.564176 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.564121 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.564326 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.564173 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kknj9\" (UniqueName: \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.564326 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.564260 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.564432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.564346 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.665923 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.665872 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4whd9\" (UniqueName: \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.666108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.665937 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.666108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.665975 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.666108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.666011 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kknj9\" (UniqueName: \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.666108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.666038 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.666108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.666101 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.667042 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.667015 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.667474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.667445 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.668469 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.668441 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="434d5008-fd97-47ad-81cf-64385b25936d" path="/var/lib/kubelet/pods/434d5008-fd97-47ad-81cf-64385b25936d/volumes" Oct 30 18:46:52.669580 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.669519 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.670881 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.670857 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c0902b1-64b3-4fae-a058-8adb0c1fc111" path="/var/lib/kubelet/pods/4c0902b1-64b3-4fae-a058-8adb0c1fc111/volumes" Oct 30 18:46:52.671673 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.671650 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed9a251c-c576-4e6c-87fa-6d295f0c7833" path="/var/lib/kubelet/pods/ed9a251c-c576-4e6c-87fa-6d295f0c7833/volumes" Oct 30 18:46:52.677121 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.677083 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.690071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.690018 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kknj9\" (UniqueName: \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\") pod \"maas-api-revoke-sub-84b9b321-59scd\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.690322 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.690298 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4whd9\" (UniqueName: \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\") pod \"maas-api-revoke-sub-58a7ada9-5jgzf\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:52.754931 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.754899 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:52.788673 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:52.788637 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:53.096616 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.096592 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd"] Oct 30 18:46:53.112202 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:46:53.112159 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f49a1e_2f82_4e75_a0b6_cef48fe34f2f.slice/crio-cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9 WatchSource:0}: Error finding container cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9: Status 404 returned error can't find the container with id cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9 Oct 30 18:46:53.148143 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.148110 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf"] Oct 30 18:46:53.748456 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.748419 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" event={"ID":"4df5834a-ec80-4238-93be-ae5fb132360f","Type":"ContainerStarted","Data":"10b82cdca4f928ce82634df178fe778982c913a9591c8845a094b630bbe82a89"} Oct 30 18:46:53.748936 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.748483 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" event={"ID":"4df5834a-ec80-4238-93be-ae5fb132360f","Type":"ContainerStarted","Data":"9b9df2f668bd91a359294c79d3f401ef5ae05cca2ae73c3a874b59dfb7d1b420"} Oct 30 18:46:53.750321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.750293 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" event={"ID":"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f","Type":"ContainerStarted","Data":"a6f858cafd5e495dbff670b32e70799e589acf65858f52a1cc97f9f0653abceb"} Oct 30 18:46:53.750428 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.750342 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" event={"ID":"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f","Type":"ContainerStarted","Data":"cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9"} Oct 30 18:46:53.791487 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.791430 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" podStartSLOduration=1.791415868 podStartE2EDuration="1.791415868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:53.789268432 +0000 UTC m=+885.233659449" watchObservedRunningTime="2026-10-30 18:46:53.791415868 +0000 UTC m=+885.235806863" Oct 30 18:46:53.835151 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:53.835087 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" podStartSLOduration=1.835072228 podStartE2EDuration="1.835072228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:53.832907289 +0000 UTC m=+885.277298284" watchObservedRunningTime="2026-10-30 18:46:53.835072228 +0000 UTC m=+885.279463263" Oct 30 18:46:54.624951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.624910 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" podUID="c965d665-eff2-4db5-bdf3-b5513145ddd2" containerName="main" containerID="cri-o://36f5f4d378b6846c80e13ef6e496bcb7af0f135150c6373b7e502bc0effb975d" gracePeriod=45 Oct 30 18:46:54.759009 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.758970 6492 generic.go:356] "Generic (PLEG): container finished" podID="c965d665-eff2-4db5-bdf3-b5513145ddd2" containerID="36f5f4d378b6846c80e13ef6e496bcb7af0f135150c6373b7e502bc0effb975d" exitCode=0 Oct 30 18:46:54.759423 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.759025 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" event={"ID":"c965d665-eff2-4db5-bdf3-b5513145ddd2","Type":"ContainerDied","Data":"36f5f4d378b6846c80e13ef6e496bcb7af0f135150c6373b7e502bc0effb975d"} Oct 30 18:46:54.761044 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.761012 6492 generic.go:356] "Generic (PLEG): container finished" podID="4df5834a-ec80-4238-93be-ae5fb132360f" containerID="10b82cdca4f928ce82634df178fe778982c913a9591c8845a094b630bbe82a89" exitCode=0 Oct 30 18:46:54.761167 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.761085 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" event={"ID":"4df5834a-ec80-4238-93be-ae5fb132360f","Type":"ContainerDied","Data":"10b82cdca4f928ce82634df178fe778982c913a9591c8845a094b630bbe82a89"} Oct 30 18:46:54.762794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.762766 6492 generic.go:356] "Generic (PLEG): container finished" podID="e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" containerID="a6f858cafd5e495dbff670b32e70799e589acf65858f52a1cc97f9f0653abceb" exitCode=0 Oct 30 18:46:54.762890 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.762835 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" event={"ID":"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f","Type":"ContainerDied","Data":"a6f858cafd5e495dbff670b32e70799e589acf65858f52a1cc97f9f0653abceb"} Oct 30 18:46:54.891417 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.891392 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:46:54.989119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989081 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\" (UniqueName: \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989288 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989146 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989288 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989250 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989294 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989358 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989432 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989397 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:54.989597 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989440 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\" (UniqueName: \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\") pod \"c965d665-eff2-4db5-bdf3-b5513145ddd2\" (UID: \"c965d665-eff2-4db5-bdf3-b5513145ddd2\") " Oct 30 18:46:54.989597 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989482 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:54.989668 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989638 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:54.989826 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989805 6492 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-tmp-dir\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:54.989886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989829 6492 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-model-cache\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:54.989886 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.989838 6492 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-home\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:54.991451 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.991428 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:54.991572 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.991455 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "kube-api-access-kgvgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:54.991572 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:54.991463 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs" pod "c965d665-eff2-4db5-bdf3-b5513145ddd2" (UID: "c965d665-eff2-4db5-bdf3-b5513145ddd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:55.091352 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.091311 6492 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c965d665-eff2-4db5-bdf3-b5513145ddd2-tls-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:55.091352 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.091344 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgvgh\" (UniqueName: \"kubernetes.io/projected/c965d665-eff2-4db5-bdf3-b5513145ddd2-kube-api-access-kgvgh\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:55.091352 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.091358 6492 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c965d665-eff2-4db5-bdf3-b5513145ddd2-dshm\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:55.771000 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.770967 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" Oct 30 18:46:55.771472 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.771003 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892" event={"ID":"c965d665-eff2-4db5-bdf3-b5513145ddd2","Type":"ContainerDied","Data":"27e2c31d7bfa4ef7c242b7cc0aa2574cd814641daa4c44a3393ce7bf1656e655"} Oct 30 18:46:55.771472 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.771049 6492 scope.go:122] "RemoveContainer" containerID="36f5f4d378b6846c80e13ef6e496bcb7af0f135150c6373b7e502bc0effb975d" Oct 30 18:46:55.789198 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.789165 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" containerID="cri-o://6bd46fccaa1c7b10284e71c2d794e06d105f5c1f918e633eceae0ed8782d1d34" gracePeriod=25 Oct 30 18:46:55.838106 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.838075 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:46:55.845326 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.845294 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-a-6c6935b2-kserve-c6c964d4f-mt892"] Oct 30 18:46:55.924240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.924184 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:55.953947 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:55.953924 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:56.008515 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.008477 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\" (UniqueName: \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\") pod \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " Oct 30 18:46:56.008732 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.008560 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\") pod \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " Oct 30 18:46:56.008732 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.008606 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\") pod \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\" (UID: \"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f\") " Oct 30 18:46:56.008920 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.008898 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca" pod "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" (UID: "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:56.010710 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.010687 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca" pod "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" (UID: "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:56.010854 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.010775 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9" pod "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" (UID: "e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f"). InnerVolumeSpecName "kube-api-access-kknj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.109751 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.109715 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\") pod \"4df5834a-ec80-4238-93be-ae5fb132360f\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " Oct 30 18:46:56.109911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.109789 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\" (UniqueName: \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\") pod \"4df5834a-ec80-4238-93be-ae5fb132360f\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " Oct 30 18:46:56.109911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.109893 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\") pod \"4df5834a-ec80-4238-93be-ae5fb132360f\" (UID: \"4df5834a-ec80-4238-93be-ae5fb132360f\") " Oct 30 18:46:56.110088 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.110075 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kknj9\" (UniqueName: \"kubernetes.io/projected/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-kube-api-access-kknj9\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.110148 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.110088 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.110148 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.110099 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.110305 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.110280 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca" pod "4df5834a-ec80-4238-93be-ae5fb132360f" (UID: "4df5834a-ec80-4238-93be-ae5fb132360f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:56.111900 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.111872 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9" pod "4df5834a-ec80-4238-93be-ae5fb132360f" (UID: "4df5834a-ec80-4238-93be-ae5fb132360f"). InnerVolumeSpecName "kube-api-access-4whd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.111900 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.111884 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca" pod "4df5834a-ec80-4238-93be-ae5fb132360f" (UID: "4df5834a-ec80-4238-93be-ae5fb132360f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:56.211501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.211473 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-openshift-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.211501 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.211500 6492 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4df5834a-ec80-4238-93be-ae5fb132360f-maas-api-xks-ca\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.211701 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.211510 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4whd9\" (UniqueName: \"kubernetes.io/projected/4df5834a-ec80-4238-93be-ae5fb132360f-kube-api-access-4whd9\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.271109 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.271083 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:46:56.412864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.412771 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.412864 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.412821 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.412872 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.412930 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.412978 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413021 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413100 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413058 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413052 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413138 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413147 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413262 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\") pod \"e66bbe00-2144-4e59-9b40-de0e73000557\" (UID: \"e66bbe00-2144-4e59-9b40-de0e73000557\") " Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413270 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.413341 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413312 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.413635 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413349 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:56.413687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413642 6492 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-credential-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.413687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413664 6492 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.413687 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413677 6492 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-workload-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.413807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413691 6492 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-data\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.413807 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.413722 6492 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66bbe00-2144-4e59-9b40-de0e73000557-istiod-ca-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.415239 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.415218 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.415331 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.415280 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.415719 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.415677 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:56.415719 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.415699 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8" pod "e66bbe00-2144-4e59-9b40-de0e73000557" (UID: "e66bbe00-2144-4e59-9b40-de0e73000557"). InnerVolumeSpecName "kube-api-access-lvfp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.515141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.515107 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvfp8\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-kube-api-access-lvfp8\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.515141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.515136 6492 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66bbe00-2144-4e59-9b40-de0e73000557-istio-token\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.515141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.515146 6492 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66bbe00-2144-4e59-9b40-de0e73000557-istio-podinfo\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.515141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.515155 6492 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66bbe00-2144-4e59-9b40-de0e73000557-istio-envoy\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.664582 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.664499 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c965d665-eff2-4db5-bdf3-b5513145ddd2" path="/var/lib/kubelet/pods/c965d665-eff2-4db5-bdf3-b5513145ddd2/volumes" Oct 30 18:46:56.778084 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.778048 6492 generic.go:356] "Generic (PLEG): container finished" podID="e66bbe00-2144-4e59-9b40-de0e73000557" containerID="2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58" exitCode=0 Oct 30 18:46:56.778569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.778096 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" Oct 30 18:46:56.778569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.778140 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" event={"ID":"e66bbe00-2144-4e59-9b40-de0e73000557","Type":"ContainerDied","Data":"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58"} Oct 30 18:46:56.778569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.778203 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t" event={"ID":"e66bbe00-2144-4e59-9b40-de0e73000557","Type":"ContainerDied","Data":"e14ecb26f25afa029ae5c4eb9ef44e168536cf932edb82e26db1a3be39bca9f3"} Oct 30 18:46:56.778569 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.778230 6492 scope.go:122] "RemoveContainer" containerID="2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58" Oct 30 18:46:56.779660 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.779635 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" Oct 30 18:46:56.779740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.779634 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a7ada9-5jgzf" event={"ID":"4df5834a-ec80-4238-93be-ae5fb132360f","Type":"ContainerDied","Data":"9b9df2f668bd91a359294c79d3f401ef5ae05cca2ae73c3a874b59dfb7d1b420"} Oct 30 18:46:56.779789 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.779771 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b9df2f668bd91a359294c79d3f401ef5ae05cca2ae73c3a874b59dfb7d1b420" Oct 30 18:46:56.781413 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.781398 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" Oct 30 18:46:56.781511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.781432 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-84b9b321-59scd" event={"ID":"e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f","Type":"ContainerDied","Data":"cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9"} Oct 30 18:46:56.781511 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.781471 6492 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cac03a42e2646bb5cda34630ebdb3403726d2a3767491bd7fedfaa2b734e5ad9" Oct 30 18:46:56.789607 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.789586 6492 scope.go:122] "RemoveContainer" containerID="2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58" Oct 30 18:46:56.789885 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:46:56.789867 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58\": container with ID starting with 2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58 not found: ID does not exist" containerID="2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58" Oct 30 18:46:56.789944 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.789898 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58"} err="failed to get container status \"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58\": rpc error: code = NotFound desc = could not find container \"2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58\": container with ID starting with 2a924ee71942fda505bf9834f2bddc0aadbbbb59a5d7cc27a8956d45fae71b58 not found: ID does not exist" Oct 30 18:46:56.832997 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.832964 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:46:56.844650 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:56.844618 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-h8h4t"] Oct 30 18:46:57.014426 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:57.014389 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:58.665908 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:58.665874 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" path="/var/lib/kubelet/pods/e66bbe00-2144-4e59-9b40-de0e73000557/volumes" Oct 30 18:46:59.945026 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:46:59.944980 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:47:05.526141 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.526116 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:47:05.713375 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.713342 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\" (UniqueName: \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\") pod \"a97bb660-8c09-4628-b654-144b0bf021cd\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " Oct 30 18:47:05.713574 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.713465 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\") pod \"a97bb660-8c09-4628-b654-144b0bf021cd\" (UID: \"a97bb660-8c09-4628-b654-144b0bf021cd\") " Oct 30 18:47:05.713860 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.713836 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume" pod "a97bb660-8c09-4628-b654-144b0bf021cd" (UID: "a97bb660-8c09-4628-b654-144b0bf021cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:05.715520 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.715499 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm" pod "a97bb660-8c09-4628-b654-144b0bf021cd" (UID: "a97bb660-8c09-4628-b654-144b0bf021cd"). InnerVolumeSpecName "kube-api-access-cfxbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:05.815557 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.815421 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a97bb660-8c09-4628-b654-144b0bf021cd-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.815720 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.815558 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfxbm\" (UniqueName: \"kubernetes.io/projected/a97bb660-8c09-4628-b654-144b0bf021cd-kube-api-access-cfxbm\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.846794 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.846756 6492 generic.go:356] "Generic (PLEG): container finished" podID="a97bb660-8c09-4628-b654-144b0bf021cd" containerID="af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0" exitCode=0 Oct 30 18:47:05.846956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.846813 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" Oct 30 18:47:05.846956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.846848 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" event={"ID":"a97bb660-8c09-4628-b654-144b0bf021cd","Type":"ContainerDied","Data":"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0"} Oct 30 18:47:05.846956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.846917 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt" event={"ID":"a97bb660-8c09-4628-b654-144b0bf021cd","Type":"ContainerDied","Data":"1b6cc717aa361f1b6f45aabe08d981361f7263f3b2f53b117d7acb89d43573d5"} Oct 30 18:47:05.846956 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.846945 6492 scope.go:122] "RemoveContainer" containerID="af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0" Oct 30 18:47:05.859289 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.859261 6492 scope.go:122] "RemoveContainer" containerID="af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0" Oct 30 18:47:05.859593 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:47:05.859568 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0\": container with ID starting with af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0 not found: ID does not exist" containerID="af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0" Oct 30 18:47:05.859645 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.859604 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0"} err="failed to get container status \"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0\": rpc error: code = NotFound desc = could not find container \"af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0\": container with ID starting with af70f1244a977829f20968c92c4f406294bb5b4a3907e88de3d0a3a07aefd0d0 not found: ID does not exist" Oct 30 18:47:05.901301 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.901268 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:47:05.914736 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:05.914696 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-a759a1-6bd858c9f6-rcpbt"] Oct 30 18:47:06.122877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:06.122786 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh"] Oct 30 18:47:06.128799 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:06.128765 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-0ddc26-dx9mh"] Oct 30 18:47:06.664593 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:06.664559 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4659b622-5e9f-4922-9327-b7d95380b972" path="/var/lib/kubelet/pods/4659b622-5e9f-4922-9327-b7d95380b972/volumes" Oct 30 18:47:06.665071 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:06.664936 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" path="/var/lib/kubelet/pods/a97bb660-8c09-4628-b654-144b0bf021cd/volumes" Oct 30 18:47:08.808414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:08.808378 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:47:08.814584 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:08.814556 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6v4xj_dbee4f62-3043-46ce-821a-b77f3746f83a/ovn-acl-logging/0.log" Oct 30 18:47:09.944491 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:09.944454 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:47:13.482665 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:13.482625 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:47:13.483159 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:13.482880 6492 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" containerID="cri-o://236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc" gracePeriod=30 Oct 30 18:47:14.313770 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:14.313725 6492 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:47:14.313951 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:14.313788 6492 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:47:14.630984 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:14.630894 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:47:15.909596 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:15.909559 6492 generic.go:356] "Generic (PLEG): container finished" podID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerID="6bd46fccaa1c7b10284e71c2d794e06d105f5c1f918e633eceae0ed8782d1d34" exitCode=0 Oct 30 18:47:15.910056 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:15.909714 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" event={"ID":"b3243bc3-ea41-48cf-9304-ee914c9aec07","Type":"ContainerDied","Data":"6bd46fccaa1c7b10284e71c2d794e06d105f5c1f918e633eceae0ed8782d1d34"} Oct 30 18:47:16.049713 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.049642 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:47:16.108842 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.108809 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\") pod \"b3243bc3-ea41-48cf-9304-ee914c9aec07\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " Oct 30 18:47:16.109045 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.108941 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\" (UniqueName: \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\") pod \"b3243bc3-ea41-48cf-9304-ee914c9aec07\" (UID: \"b3243bc3-ea41-48cf-9304-ee914c9aec07\") " Oct 30 18:47:16.109250 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.109223 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume" pod "b3243bc3-ea41-48cf-9304-ee914c9aec07" (UID: "b3243bc3-ea41-48cf-9304-ee914c9aec07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:16.111109 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.111085 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d" pod "b3243bc3-ea41-48cf-9304-ee914c9aec07" (UID: "b3243bc3-ea41-48cf-9304-ee914c9aec07"). InnerVolumeSpecName "kube-api-access-q499d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:16.209800 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.209756 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q499d\" (UniqueName: \"kubernetes.io/projected/b3243bc3-ea41-48cf-9304-ee914c9aec07-kube-api-access-q499d\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:16.209800 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.209792 6492 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3243bc3-ea41-48cf-9304-ee914c9aec07-plugins-config-volume\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:16.903760 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.903724 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:47:16.917256 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.917221 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" event={"ID":"b3243bc3-ea41-48cf-9304-ee914c9aec07","Type":"ContainerDied","Data":"8b37f3aba5b2fd5da5823e2ae3c0c156a25aedf979843082049d333655970f4a"} Oct 30 18:47:16.917719 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.917280 6492 scope.go:122] "RemoveContainer" containerID="6bd46fccaa1c7b10284e71c2d794e06d105f5c1f918e633eceae0ed8782d1d34" Oct 30 18:47:16.917719 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.917234 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq" Oct 30 18:47:16.971703 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.971666 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:47:16.973780 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:47:16.973736 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:47:16.983945 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:16.983921 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-0ddc26-6dbbc755cf-rtlmq"] Oct 30 18:47:18.593164 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.593130 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:47:18.670085 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.670052 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" path="/var/lib/kubelet/pods/b3243bc3-ea41-48cf-9304-ee914c9aec07/volumes" Oct 30 18:47:18.753108 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.753083 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:47:18.809306 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.809176 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:47:18.836096 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836058 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836110 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836142 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836204 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836242 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836290 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836270 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836300 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836356 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836412 6492 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\") pod \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\" (UID: \"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff\") " Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836423 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836462 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.836577 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836472 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.836877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836741 6492 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-credential-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.836877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836761 6492 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-socket\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.836877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836775 6492 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-workload-certs\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.836877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.836807 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:18.837073 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.837010 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.838837 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.838812 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.839059 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.839041 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.839135 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.839087 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:47:18.839883 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.839861 6492 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv" pod "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" (UID: "97c9e05a-cef4-4cba-a30b-e4efa8ec36ff"). InnerVolumeSpecName "kube-api-access-j6fbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.932247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.932170 6492 generic.go:356] "Generic (PLEG): container finished" podID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerID="236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc" exitCode=0 Oct 30 18:47:18.932247 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.932222 6492 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" Oct 30 18:47:18.932460 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.932264 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" event={"ID":"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff","Type":"ContainerDied","Data":"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc"} Oct 30 18:47:18.932460 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.932331 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g" event={"ID":"97c9e05a-cef4-4cba-a30b-e4efa8ec36ff","Type":"ContainerDied","Data":"ff6fd2df2fc4b121b09b3e791cdd1745621620458082bc00bc4693118c3eb087"} Oct 30 18:47:18.932460 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.932362 6492 scope.go:122] "RemoveContainer" containerID="236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939048 6492 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-podinfo\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939088 6492 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-data\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939104 6492 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-envoy\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939119 6492 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istio-token\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939133 6492 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6fbv\" (UniqueName: \"kubernetes.io/projected/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-kube-api-access-j6fbv\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.940468 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.939152 6492 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff-istiod-ca-cert\") on node \"ip-10-0-136-105.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.952013 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.951561 6492 scope.go:122] "RemoveContainer" containerID="236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc" Oct 30 18:47:18.952013 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:47:18.951916 6492 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc\": container with ID starting with 236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc not found: ID does not exist" containerID="236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc" Oct 30 18:47:18.952013 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.951953 6492 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc"} err="failed to get container status \"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc\": rpc error: code = NotFound desc = could not find container \"236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc\": container with ID starting with 236d62211cab41e40d88fcf7ae1343e7bf4d0dee1fbc4c0a8d4a5de7041887cc not found: ID does not exist" Oct 30 18:47:18.989810 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.989770 6492 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:47:18.994181 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:18.994152 6492 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-w9w2g"] Oct 30 18:47:19.025851 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:19.025818 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:47:19.682745 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:19.682714 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:47:19.919699 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:19.919667 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:47:20.664236 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:20.664200 6492 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" path="/var/lib/kubelet/pods/97c9e05a-cef4-4cba-a30b-e4efa8ec36ff/volumes" Oct 30 18:47:21.144626 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:21.144592 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:47:32.734663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:32.734628 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:47:32.957713 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:32.957684 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:47:33.195177 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:33.195075 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:47:33.749105 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:33.749077 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:47:33.957634 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:33.957608 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:47:35.399931 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:35.399896 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-1085ab69-8h6jl_b94888d6-adbd-4f7e-be24-6e6bd89d94c9/revoke-keys/0.log" Oct 30 18:47:35.609686 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:35.609657 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-189cb936-qqj5c_f28daf6b-fee5-4a60-bc85-88672e5681eb/revoke-keys/0.log" Oct 30 18:47:35.805249 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:35.805216 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3524ffb7-rdhks_1fb286c0-b48e-4c0a-a2e4-445d8c0008fc/revoke-keys/0.log" Oct 30 18:47:36.025932 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:36.025903 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3a794a61-4j9mq_077cdcb0-6ac6-4f73-b65b-95ba34759d6d/revoke-keys/0.log" Oct 30 18:47:36.221474 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:36.221445 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-49f32b28-vf6bq_b5105898-a20a-49bc-ae82-2709062aad76/revoke-keys/0.log" Oct 30 18:47:36.597855 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:36.597776 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-504b60eb-xkdwc_c63a7d71-9899-4408-810c-7d3fd1750c44/revoke-keys/0.log" Oct 30 18:47:36.794894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:36.794863 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-58a7ada9-5jgzf_4df5834a-ec80-4238-93be-ae5fb132360f/revoke-keys/0.log" Oct 30 18:47:36.998005 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:36.997975 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-59766bc4-h9mj4_7070e575-6253-48fe-9325-8450c80fa6dd/revoke-keys/0.log" Oct 30 18:47:37.187473 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:37.187438 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-59c67ac2-5m4ts_cd219385-41d8-4891-948b-aa269703a45c/revoke-keys/0.log" Oct 30 18:47:37.389985 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:37.389910 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-65d74124-mnsq9_541dd9ad-1acf-4edf-8a72-6bbe46b28c6c/revoke-keys/0.log" Oct 30 18:47:37.589319 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:37.589278 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6991f792-tw6w6_1df424f9-f44b-4a86-a0a9-9b5ed0e6f352/revoke-keys/0.log" Oct 30 18:47:37.997171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:37.997136 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-84b9b321-59scd_e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f/revoke-keys/0.log" Oct 30 18:47:38.187301 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:38.187272 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8806e007-wzkbv_0d516a63-65f2-474a-97e2-7188bd728a8d/revoke-keys/0.log" Oct 30 18:47:38.390735 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:38.390642 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8b86648d-d6m6j_e530a0b4-122a-4223-bd2d-1677ba0754ec/revoke-keys/0.log" Oct 30 18:47:38.788076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:38.788049 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c188a728-76t5x_3c6eaee4-f6f0-4c65-9152-e400c9153d15/revoke-keys/0.log" Oct 30 18:47:38.986694 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:38.986665 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f582a077-m5bjm_4cba3a53-46df-421e-b546-bc7cef00248a/revoke-keys/0.log" Oct 30 18:47:39.182990 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:39.182914 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f8c43280-mfscv_74bfd517-a7cd-4b83-a3e4-a28e53fd2bf4/revoke-keys/0.log" Oct 30 18:47:41.019168 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:41.019139 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:47:42.497367 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:42.497330 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:47:44.654940 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:44.654905 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:47:44.864102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:44.864069 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:47:46.320347 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:46.320314 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:47:46.727389 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:46.727356 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:47:46.971878 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:47:46.971837 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:47:48.007831 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:47:48.007804 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:00.470313 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470279 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ggghx/must-gather-x28td"] Oct 30 18:48:00.470766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470741 6492 cpu_manager.go:414] "containerMap: removing container" podUID="e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" containerName="revoke-keys" Oct 30 18:48:00.470766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470754 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" containerName="revoke-keys" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470770 6492 cpu_manager.go:414] "containerMap: removing container" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470775 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470784 6492 cpu_manager.go:414] "containerMap: removing container" podUID="4df5834a-ec80-4238-93be-ae5fb132360f" containerName="revoke-keys" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470789 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4df5834a-ec80-4238-93be-ae5fb132360f" containerName="revoke-keys" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470797 6492 cpu_manager.go:414] "containerMap: removing container" podUID="c965d665-eff2-4db5-bdf3-b5513145ddd2" containerName="main" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470803 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c965d665-eff2-4db5-bdf3-b5513145ddd2" containerName="main" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470812 6492 cpu_manager.go:414] "containerMap: removing container" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470816 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470830 6492 cpu_manager.go:414] "containerMap: removing container" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" Oct 30 18:48:00.470846 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470835 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470856 6492 cpu_manager.go:414] "containerMap: removing container" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470860 6492 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470915 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="e66bbe00-2144-4e59-9b40-de0e73000557" containerName="istio-proxy" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470923 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3f49a1e-2f82-4e75-a0b6-cef48fe34f2f" containerName="revoke-keys" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470929 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="a97bb660-8c09-4628-b654-144b0bf021cd" containerName="payload-processing" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470935 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="4df5834a-ec80-4238-93be-ae5fb132360f" containerName="revoke-keys" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470939 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="c965d665-eff2-4db5-bdf3-b5513145ddd2" containerName="main" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470945 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="97c9e05a-cef4-4cba-a30b-e4efa8ec36ff" containerName="istio-proxy" Oct 30 18:48:00.471131 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.470951 6492 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3243bc3-ea41-48cf-9304-ee914c9aec07" containerName="payload-pre-processing" Oct 30 18:48:00.474257 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.474235 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.497310 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.497263 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ggghx\"/\"openshift-service-ca.crt\"" Oct 30 18:48:00.523252 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.523214 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ggghx\"/\"kube-root-ca.crt\"" Oct 30 18:48:00.523471 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.523390 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ggghx\"/\"default-dockercfg-dzmwp\"" Oct 30 18:48:00.526240 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.526212 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ggghx/must-gather-x28td"] Oct 30 18:48:00.538692 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.538664 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d30b547f-b504-4705-8548-a86397890f26-must-gather-output\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.538814 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.538713 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xwvd\" (UniqueName: \"kubernetes.io/projected/d30b547f-b504-4705-8548-a86397890f26-kube-api-access-6xwvd\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.639980 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.639939 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d30b547f-b504-4705-8548-a86397890f26-must-gather-output\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.640169 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.640001 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xwvd\" (UniqueName: \"kubernetes.io/projected/d30b547f-b504-4705-8548-a86397890f26-kube-api-access-6xwvd\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.640321 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.640300 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d30b547f-b504-4705-8548-a86397890f26-must-gather-output\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.665147 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.665116 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xwvd\" (UniqueName: \"kubernetes.io/projected/d30b547f-b504-4705-8548-a86397890f26-kube-api-access-6xwvd\") pod \"must-gather-x28td\" (UID: \"d30b547f-b504-4705-8548-a86397890f26\") " pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:00.789142 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:00.789041 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ggghx/must-gather-x28td" Oct 30 18:48:01.099149 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:01.099120 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ggghx/must-gather-x28td"] Oct 30 18:48:01.101194 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:48:01.101166 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd30b547f_b504_4705_8548_a86397890f26.slice/crio-570c49b63cd54fe821c10dd40b7f3422146e8af5c8fba980df3127c1f1c493ea WatchSource:0}: Error finding container 570c49b63cd54fe821c10dd40b7f3422146e8af5c8fba980df3127c1f1c493ea: Status 404 returned error can't find the container with id 570c49b63cd54fe821c10dd40b7f3422146e8af5c8fba980df3127c1f1c493ea Oct 30 18:48:01.103063 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:01.103047 6492 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:48:01.194229 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:01.194189 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ggghx/must-gather-x28td" event={"ID":"d30b547f-b504-4705-8548-a86397890f26","Type":"ContainerStarted","Data":"570c49b63cd54fe821c10dd40b7f3422146e8af5c8fba980df3127c1f1c493ea"} Oct 30 18:48:05.286251 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:05.286220 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t6kkd_a56c3ed8-1972-402c-8195-b4dc842c69fb/global-pull-secret-syncer/0.log" Oct 30 18:48:06.013137 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:06.013107 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rl74h_ba5fa788-50a7-47bb-bfb3-231034995cea/konnectivity-agent/0.log" Oct 30 18:48:06.234658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:06.234573 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ggghx/must-gather-x28td" event={"ID":"d30b547f-b504-4705-8548-a86397890f26","Type":"ContainerStarted","Data":"932f9bd8c4fa2cff23dbfd817cfebc5891eb3b095b61baeb5f90952cf0abf756"} Oct 30 18:48:06.234658 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:06.234616 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ggghx/must-gather-x28td" event={"ID":"d30b547f-b504-4705-8548-a86397890f26","Type":"ContainerStarted","Data":"65ff0811f22d7b6eee5e807c2e25ec0d443c2ed96d172a1c49c97662f25ff4a0"} Oct 30 18:48:06.287935 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:06.287881 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-ggghx/must-gather-x28td" podStartSLOduration=1.419925506 podStartE2EDuration="6.287864909s" totalImagesPullingTime="4.867939403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:06.273961377 +0000 UTC m=+957.718352370" watchObservedRunningTime="2026-10-30 18:48:06.287864909 +0000 UTC m=+957.732255903" Oct 30 18:48:06.496631 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:06.496527 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-105.ec2.internal_c2a961c4fb7c61fa12bec5d71405aa03/haproxy/0.log" Oct 30 18:48:07.791600 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:07.791568 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:07.857763 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:07.857734 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:07.894298 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:07.894269 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:07.961818 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:07.961785 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:07.974517 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:07.974489 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:08.187915 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.187823 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:08.261399 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.261367 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:08.426989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.426963 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:08.503986 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.503949 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:08.691813 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.691785 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:08.925296 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.925220 6492 scope.go:122] "RemoveContainer" containerID="08f9767608721878f98a679761191265d07a8b9bc7331ab0e2ae6b0aa11d81a2" Oct 30 18:48:08.934534 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.934512 6492 scope.go:122] "RemoveContainer" containerID="fa522dfca07c5c04b7feb3031c3cfe3cd1dcbbd99d334ede59aac761827de39e" Oct 30 18:48:08.942814 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.942793 6492 scope.go:122] "RemoveContainer" containerID="85c2eb70f55389312f7d88fd82481bbe0b20121a4d878123784f96cd2bb17650" Oct 30 18:48:08.951119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:08.951097 6492 scope.go:122] "RemoveContainer" containerID="2f42edaadd08f93d89eccf3ed385050074b88a2d3295ab41abfc38ad5c60dc9b" Oct 30 18:48:09.912647 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:09.912616 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:09.977816 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:09.977784 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:10.101236 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.101204 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:10.189877 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.189807 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:10.260857 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.260819 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:10.340882 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.340854 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:10.392838 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.392808 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:10.552666 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.552627 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:10.608512 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.608482 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:10.886248 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:10.886158 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:12.047522 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.047495 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:12.113964 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.113935 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:12.233894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.233842 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:12.333780 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.333700 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:12.407153 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.407108 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:12.441354 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.441320 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:12.557427 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.557379 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:12.654119 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.654049 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:12.794937 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.794906 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:12.991913 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:12.991882 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:14.945989 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:14.945957 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:15.010828 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.010797 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:15.011749 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.011717 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:15.167064 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.167032 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:15.230803 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.230773 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:15.231588 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.231571 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:15.445905 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.445857 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:15.577480 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.577404 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:15.664740 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.664710 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:15.801171 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:15.801137 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:16.983419 ip-10-0-136-105 kubenswrapper[6492]: E1030 18:48:16.983308 6492 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-w9swt" containerName="extractor" Oct 30 18:48:18.061201 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:18.061163 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:48:18.249214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:18.249175 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:48:18.522764 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:18.522731 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:48:18.906096 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:18.905998 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:48:19.296520 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:19.296484 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:48:19.484234 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:19.484200 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:48:20.116797 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.116758 6492 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f"] Oct 30 18:48:20.122092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.122072 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.142162 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.142136 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f"] Oct 30 18:48:20.157498 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.157476 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"openshift-service-ca.crt\"" Oct 30 18:48:20.157704 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.157565 6492 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"kube-root-ca.crt\"" Oct 30 18:48:20.181024 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.181001 6492 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6tq9m\"/\"default-dockercfg-vlv4w\"" Oct 30 18:48:20.249243 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.249212 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wlxk\" (UniqueName: \"kubernetes.io/projected/77da64f3-afe7-4f73-8daf-5941cc0b27af-kube-api-access-9wlxk\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.249414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.249260 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-podres\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.249414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.249289 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-sys\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.249414 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.249381 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-proc\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.249526 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.249504 6492 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-lib-modules\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350118 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350085 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-podres\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350135 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-sys\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350170 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-proc\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350211 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-podres\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350249 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-lib-modules\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350278 6492 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wlxk\" (UniqueName: \"kubernetes.io/projected/77da64f3-afe7-4f73-8daf-5941cc0b27af-kube-api-access-9wlxk\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350289 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-sys\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350338 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-proc\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.350604 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.350346 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77da64f3-afe7-4f73-8daf-5941cc0b27af-lib-modules\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.374916 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.374841 6492 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wlxk\" (UniqueName: \"kubernetes.io/projected/77da64f3-afe7-4f73-8daf-5941cc0b27af-kube-api-access-9wlxk\") pod \"perf-node-gather-daemonset-kmc7f\" (UID: \"77da64f3-afe7-4f73-8daf-5941cc0b27af\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.434495 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.434462 6492 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:20.742394 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:20.742368 6492 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f"] Oct 30 18:48:20.743218 ip-10-0-136-105 kubenswrapper[6492]: W1030 18:48:20.743188 6492 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77da64f3_afe7_4f73_8daf_5941cc0b27af.slice/crio-0cc36c6c5beda03d444d47b3e91120a101d83f9ed40eb132b29d8bb896f7e15e WatchSource:0}: Error finding container 0cc36c6c5beda03d444d47b3e91120a101d83f9ed40eb132b29d8bb896f7e15e: Status 404 returned error can't find the container with id 0cc36c6c5beda03d444d47b3e91120a101d83f9ed40eb132b29d8bb896f7e15e Oct 30 18:48:21.336222 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:21.336184 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" event={"ID":"77da64f3-afe7-4f73-8daf-5941cc0b27af","Type":"ContainerStarted","Data":"7572ae71a9fc4af74d606cb554a9c4261a36bb46aecc98cb4b893f8086147775"} Oct 30 18:48:21.336410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:21.336250 6492 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" event={"ID":"77da64f3-afe7-4f73-8daf-5941cc0b27af","Type":"ContainerStarted","Data":"0cc36c6c5beda03d444d47b3e91120a101d83f9ed40eb132b29d8bb896f7e15e"} Oct 30 18:48:21.336410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:21.336277 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:21.377894 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:21.377845 6492 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" podStartSLOduration=1.3778310999999999 podStartE2EDuration="1.3778311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:21.376442377 +0000 UTC m=+972.820833375" watchObservedRunningTime="2026-10-30 18:48:21.3778311 +0000 UTC m=+972.822222133" Oct 30 18:48:22.215263 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:22.215236 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7969db6d87-blhz9_852071ed-f1b5-4b6e-8ef5-1deb88d86855/metrics-server/0.log" Oct 30 18:48:22.440960 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:22.440925 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24l9n_3dc7152c-07ad-466c-aed1-1e289c384fde/node-exporter/0.log" Oct 30 18:48:22.550723 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:22.550645 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24l9n_3dc7152c-07ad-466c-aed1-1e289c384fde/kube-rbac-proxy/0.log" Oct 30 18:48:22.662984 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:22.662955 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24l9n_3dc7152c-07ad-466c-aed1-1e289c384fde/init-textfile/0.log" Oct 30 18:48:23.403963 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.403934 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-w6tdb_4bd5c23b-225b-4eff-8b03-77fe7c04e56c/kube-rbac-proxy-main/0.log" Oct 30 18:48:23.517208 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.517181 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-w6tdb_4bd5c23b-225b-4eff-8b03-77fe7c04e56c/kube-rbac-proxy-self/0.log" Oct 30 18:48:23.627466 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.627418 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-w6tdb_4bd5c23b-225b-4eff-8b03-77fe7c04e56c/openshift-state-metrics/0.log" Oct 30 18:48:23.752957 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.752932 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/prometheus/0.log" Oct 30 18:48:23.863060 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.863029 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/config-reloader/0.log" Oct 30 18:48:23.975638 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:23.975609 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/thanos-sidecar/0.log" Oct 30 18:48:24.090092 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:24.090000 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/kube-rbac-proxy-web/0.log" Oct 30 18:48:24.205102 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:24.205073 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/kube-rbac-proxy/0.log" Oct 30 18:48:24.315201 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:24.315173 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/kube-rbac-proxy-thanos/0.log" Oct 30 18:48:24.426663 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:24.426595 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_79454a9c-ca82-4d0e-ba17-e2375e945f46/init-config-reloader/0.log" Oct 30 18:48:25.302036 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.302008 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/thanos-query/0.log" Oct 30 18:48:25.415600 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.415526 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/kube-rbac-proxy-web/0.log" Oct 30 18:48:25.527479 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.527448 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/kube-rbac-proxy/0.log" Oct 30 18:48:25.640564 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.640481 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/prom-label-proxy/0.log" Oct 30 18:48:25.763436 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.763406 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/kube-rbac-proxy-rules/0.log" Oct 30 18:48:25.768671 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.768647 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-webhook-5dbbbf75cc-rhkgh_4d507704-680c-4354-9395-da664ac20a6b/cert-manager-webhook/0.log" Oct 30 18:48:25.883866 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:25.883840 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67ddcc4d7f-rndt7_6c2051f0-a1e2-4a95-a7a6-9a6d45c01c0b/kube-rbac-proxy-metrics/0.log" Oct 30 18:48:26.106487 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:26.106454 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-webhook-5dbbbf75cc-rhkgh_4d507704-680c-4354-9395-da664ac20a6b/cert-manager-webhook/0.log" Oct 30 18:48:27.024982 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:27.024951 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5_49d576d5-c54d-4b82-bdfa-55610d101434/manager/0.log" Oct 30 18:48:27.083173 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:27.083144 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5_49d576d5-c54d-4b82-bdfa-55610d101434/copy-manifests/0.log" Oct 30 18:48:27.231357 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:27.231316 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5_49d576d5-c54d-4b82-bdfa-55610d101434/manager/0.log" Oct 30 18:48:27.290415 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:27.290341 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-mcwz5_49d576d5-c54d-4b82-bdfa-55610d101434/copy-manifests/0.log" Oct 30 18:48:27.356783 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:27.356756 6492 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-kmc7f" Oct 30 18:48:28.029043 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.029017 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:28.100744 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.100709 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:28.224266 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.224236 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:28.245133 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.245109 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-s47vd_7a2f6848-9e7f-4ce4-be45-9fa160d4020e/manager/0.log" Oct 30 18:48:28.313944 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.313868 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-l8dsg_c01ee88b-5ace-448e-b749-0047a0b60a7e/manager/0.log" Oct 30 18:48:28.408282 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.408231 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:28.458448 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.458422 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pnb5q_a3140ad8-9fb8-45fe-9d54-fb94859c792f/manager/0.log" Oct 30 18:48:28.594766 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.594699 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jzhdq_48f7e082-b604-435a-b648-5f92ce5e4308/server/0.log" Oct 30 18:48:28.629727 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.629691 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:28.805790 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:28.805757 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-ncjwb_ad10e439-4eba-49aa-b98e-facac59da954/manager/0.log" Oct 30 18:48:29.129871 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:29.129839 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-694f69bf4f-tpldd_4cfb133a-442c-48f2-a4c0-0cb30e64f7d8/console/0.log" Oct 30 18:48:30.310144 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.310117 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:30.374048 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.374023 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:30.374817 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.374804 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:30.426784 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.426755 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:30.490230 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.490204 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:30.491050 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.491028 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:30.791022 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.790993 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:30.880214 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.880183 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:30.916345 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:30.916320 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:31.008297 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:31.008266 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:32.359629 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:32.359603 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c8dqv_502b8dd3-61d1-4a04-883e-35cf6460a661/dns/0.log" Oct 30 18:48:32.422036 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:32.422010 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-mfkvl_a858dd87-eee3-404c-8113-a8d745f50304/jobset-operator/0.log" Oct 30 18:48:32.498076 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:32.498048 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c8dqv_502b8dd3-61d1-4a04-883e-35cf6460a661/kube-rbac-proxy/0.log" Oct 30 18:48:32.543430 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:32.543402 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-mfkvl_a858dd87-eee3-404c-8113-a8d745f50304/jobset-operator/0.log" Oct 30 18:48:33.049178 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:33.049151 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9r555_e025fb6d-2541-4243-a8d3-8fc4fc5f5940/dns-node-resolver/0.log" Oct 30 18:48:34.577148 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:34.577116 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fx675_cf7a8c1d-004b-423c-8d4c-27e2b6757aae/node-ca/0.log" Oct 30 18:48:37.058404 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.058374 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:37.180099 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.180073 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:37.180914 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.180894 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:37.713669 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.713639 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:37.777314 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.777245 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:37.778045 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.778025 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:37.936000 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:37.935972 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:38.159977 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:38.159896 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:38.196410 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:38.196382 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:38.319859 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:38.319830 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:40.008772 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:40.008736 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5774ffdd7b-lhvxf_2b098fa7-4c92-4a61-a0ac-f72f5f0659aa/authorino/0.log" Oct 30 18:48:40.111844 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:40.111810 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7fvzb_11357dc0-5895-4a3d-9969-209df38fa006/serve-healthcheck-canary/0.log" Oct 30 18:48:40.343436 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:40.343355 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-fr7x6_194f4456-d1cb-449d-8b4b-66301f152cf7/manager/0.log" Oct 30 18:48:41.190753 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.190720 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-mkwbn_7acc71a4-3041-403f-a759-707008803428/kube-auth-proxy/0.log" Oct 30 18:48:41.257911 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.257878 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/0.log" Oct 30 18:48:41.258844 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.258820 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-2bv6f_ff54aab4-243f-4c0f-b941-5e2c6839ab77/istio-proxy/1.log" Oct 30 18:48:41.684618 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.684551 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tvj2n_593147ef-fd2e-4826-99a5-76094fcc971e/istio-proxy/0.log" Oct 30 18:48:41.814597 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.814565 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-wltdx_c7e64212-44f3-44fb-82f3-6f900a72084d/istio-proxy/0.log" Oct 30 18:48:41.919123 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:41.919089 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9swt_0c7fe3f2-f764-4843-8732-f5434a5f82cc/kube-rbac-proxy/0.log" Oct 30 18:48:42.028376 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:42.028349 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9swt_0c7fe3f2-f764-4843-8732-f5434a5f82cc/exporter/0.log" Oct 30 18:48:42.138458 ip-10-0-136-105 kubenswrapper[6492]: I1030 18:48:42.138429 6492 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w9swt_0c7fe3f2-f764-4843-8732-f5434a5f82cc/extractor/0.log"