Sep 10 14:11:27.928933 ip-10-0-131-209 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 14:11:28.318647 ip-10-0-131-209 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:28.318647 ip-10-0-131-209 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 14:11:28.318647 ip-10-0-131-209 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:28.318647 ip-10-0-131-209 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 14:11:28.318647 ip-10-0-131-209 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:28.319267 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.318680    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324545    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324569    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324575    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324580    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324588    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324597    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324602    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324607    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324617    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324622    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324626    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324630    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324634    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324638    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324643    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324647    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324651    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324655    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.324796 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324659    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324663    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324670    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324679    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324684    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324688    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324692    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324715    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324720    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324725    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324729    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324733    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324738    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324742    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324753    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324762    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324767    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324771    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324775    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.325441 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324780    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324785    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324789    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.324793    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325193    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325363    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325371    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325375    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325378    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325381    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325385    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325390    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325393    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325396    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325399    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325401    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325404    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325407    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325410    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325413    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.325967 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325415    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325420    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325424    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325428    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325431    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325434    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325437    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325440    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325443    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325447    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325450    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325452    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325455    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325458    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325463    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325466    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325469    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325473    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325475    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325478    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.326533 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325481    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325484    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325487    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325490    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325492    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325495    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325497    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325500    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325503    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325505    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325508    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325510    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325513    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325515    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325518    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325521    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325524    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325526    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325529    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325532    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.327055 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325534    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.325537    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326272    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326280    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326283    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326285    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326288    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326291    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326294    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326297    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326300    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326303    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326306    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326308    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326311    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326314    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326317    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326319    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326322    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326324    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.327570 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326327    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326330    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326333    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326335    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326338    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326340    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326343    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326345    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326348    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326351    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326353    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326356    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326358    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326361    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326364    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326366    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326369    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326371    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326374    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326376    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.328090 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326379    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326383    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326388    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326392    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326395    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326398    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326401    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326404    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326407    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326410    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326413    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326416    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326420    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326423    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326426    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326428    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326431    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326434    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326437    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.328599 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326439    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326442    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326445    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326448    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326450    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326453    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326456    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326459    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326462    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326465    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326468    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326471    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326473    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326476    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326478    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326481    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326483    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326486    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326489    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326491    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.329094 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326494    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326496    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326500    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326502    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326505    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326509    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326513    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326516    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326519    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326523    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326526    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326530    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326533    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326536    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326539    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326542    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326545    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326548    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326551    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.329587 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326554    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326558    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326561    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326935    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326941    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326944    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326947    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326950    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326952    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326955    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326958    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326961    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326964    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326968    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326972    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326975    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326978    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326981    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326984    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326987    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.330091 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326989    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326992    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326995    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.326997    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327000    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327003    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327005    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327008    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327010    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327013    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327016    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327019    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327022    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327025    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327027    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327030    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327034    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327038    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.330600 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327042    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327046    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327049    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327052    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327055    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327058    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327061    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327064    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327067    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327069    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327088    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327091    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327095    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327098    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327101    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327104    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327107    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327109    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327113    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327115    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.331156 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327118    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327121    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327124    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327126    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327129    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327131    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327135    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327138    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327141    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327144    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327146    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327149    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327151    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327154    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327157    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327160    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327162    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327165    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327167    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327170    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.331654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327172    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327175    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327178    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327180    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327183    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327186    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327188    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327191    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327194    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327196    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327199    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327202    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327204    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327207    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327209    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327212    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327215    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327218    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327221    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327224    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.332192 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327226    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327229    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327231    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327234    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327296    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327336    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327342    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327346    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327350    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327353    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327357    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327362    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327365    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327368    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327371    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327374    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327377    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327380    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327383    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327386    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327390    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327395    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327400    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 10 14:11:28.332740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327403    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327406    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327409    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327412    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327416    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327421    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327425    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327428    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327431    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327434    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327437    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327440    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327443    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327447    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327450    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327453    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327456    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327459    2581 flags.go:64] FLAG: --enable-server="true"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327462    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327466    2581 flags.go:64] FLAG: --event-burst="100"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327469    2581 flags.go:64] FLAG: --event-qps="50"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327471    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327475    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327478    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 10 14:11:28.333308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327481    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327484    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327487    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327490    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327493    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327496    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327502    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327505    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327508    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327511    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327514    2581 flags.go:64] FLAG: --feature-gates=""
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327518    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327521    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327525    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327529    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327532    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327535    2581 flags.go:64] FLAG: --help="false"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327537    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327540    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327543    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327546    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327550    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327553    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327556    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 14:11:28.333949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327559    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327562    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327565    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327568    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327571    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327574    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327577    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327580    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327583    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327586    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327589    2581 flags.go:64] FLAG: --lock-file=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327592    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327595    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327598    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327605    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327608    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327611    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327614    2581 flags.go:64] FLAG: --logging-format="text"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327617    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327620    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327623    2581 flags.go:64] FLAG: --manifest-url=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327627    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327632    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327635    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327639    2581 flags.go:64] FLAG: --max-pods="110"
Sep 10 14:11:28.334586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327642    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327645    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327648    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327650    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327653    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327656    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327659    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327666    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327669    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327672    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327676    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327679    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327684    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327687    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327690    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327693    2581 flags.go:64] FLAG: --port="10250"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327712    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327717    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af7cc7e5a8052034"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327722    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327726    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327729    2581 flags.go:64] FLAG: --register-node="true"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327732    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327737    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327741    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 10 14:11:28.335276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327744    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327746    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327750    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327753    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327757    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327760    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327763    2581 flags.go:64] FLAG: --runonce="false"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327766    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327769    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327772    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327775    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327778    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327781    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327784    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327788    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327790    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327793    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327796    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327799    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327802    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327805    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327808    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327813    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327816    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327819    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 14:11:28.335886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327823    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327826    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327829    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327832    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327834    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327839    2581 flags.go:64] FLAG: --v="2"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327843    2581 flags.go:64] FLAG: --version="false"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327847    2581 flags.go:64] FLAG: --vmodule=""
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327854    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.327857    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327976    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327983    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327986    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327990    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327992    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.327997    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328000    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328002    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328005    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328008    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328011    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328013    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.336515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328016    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328018    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328021    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328024    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328026    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328029    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328032    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328034    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328037    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328039    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328042    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328045    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328047    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328050    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328053    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328057    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328060    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328062    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328066    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328068    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.337096 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328071    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328075    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328078    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328080    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328083    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328086    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328088    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328091    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328094    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328096    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328100    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328102    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328105    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328108    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328110    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328113    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328116    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328118    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328121    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328123    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.337590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328126    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328129    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328131    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328134    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328137    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328139    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328142    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328146    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328148    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328151    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328154    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328161    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328164    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328168    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328170    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328173    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328175    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328178    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328181    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328183    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.338108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328186    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328189    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328191    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328195    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328200    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328203    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328206    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328209    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328212    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328215    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328217    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328220    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328223    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328225    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328228    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328230    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328233    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328236    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328238    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.338611 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328242    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328245    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328248    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328250    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328253    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328256    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328259    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.328262    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.328874    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.335575    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.335590    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.337351    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.337360    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337417    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337423    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.339110 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337426    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337430    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337433    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337436    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337438    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337441    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337444    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337447    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337449    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337452    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337456    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337460    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337463    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337466    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337469    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337472    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337475    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337478    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337481    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.339484 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337483    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337486    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337489    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337491    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337494    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337497    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337499    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337502    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337507    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337510    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337512    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337515    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337518    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337520    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337523    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337526    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337529    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337531    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337534    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337537    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.339974 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337539    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337543    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337547    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337550    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337553    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337556    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337559    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337562    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337565    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337567    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337570    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337573    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337575    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337578    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337581    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337584    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337586    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337589    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337592    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.340515 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337595    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337598    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337602    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337605    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337607    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337610    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337613    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337615    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337618    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337620    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337623    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337626    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337628    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337631    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337633    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337637    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337641    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337644    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337647    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.340995 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337649    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337652    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337654    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337657    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337660    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337663    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337665    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337668    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337670    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337673    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337676    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337678    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337681    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337684    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337686    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337689    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337692    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337695    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337714    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.341464 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337717    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.337722    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337828    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337832    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337836    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337839    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337842    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337845    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337848    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337850    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337853    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337856    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337859    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337861    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:28.341943 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337864    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337867    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337870    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337872    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337876    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337878    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337881    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337884    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337887    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337889    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337892    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337895    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337897    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337900    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337903    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337905    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337909    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337911    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337914    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337917    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:28.342310 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337919    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337922    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337924    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337927    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337929    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337932    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337934    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337937    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337939    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337942    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337944    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337947    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337950    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337952    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337955    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337957    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337960    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337963    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337966    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337968    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:28.342814 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337971    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337974    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337976    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337979    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337981    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337984    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337987    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337990    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337992    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337995    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.337998    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338000    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338003    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338005    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338008    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338010    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338013    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338015    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338018    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338020    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:28.343300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338055    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338058    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338061    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338064    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338066    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338069    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338079    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338082    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338085    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338089    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338093    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338096    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338100    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338104    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338107    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338110    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338113    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338116    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338119    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:28.343831 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338121    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338124    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338128    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338133    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338135    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338138    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338141    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:28.338143    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.338148    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.338243    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.340355    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.341285    2581 server.go:1019] "Starting client certificate rotation"
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.341387    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:11:28.344303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.341421    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:11:28.344627 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.343744    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:11:28.348479 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.348464    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:11:28.359641 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.359624    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 10 14:11:28.363159 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.363144    2581 log.go:25] "Validated CRI v1 image API"
Sep 10 14:11:28.364003 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.363989    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 14:11:28.367243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.367225    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bcbead22-f2d1-4932-99fc-1a6f3013fa54:/dev/nvme0n1p3 dba97771-7042-4c02-99b2-2ba6d7325957:/dev/nvme0n1p4]
Sep 10 14:11:28.367298 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.367244    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 14:11:28.372161 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.372052    2581 manager.go:216] Machine: {Timestamp:2026-09-10 14:11:28.370817047 +0000 UTC m=+0.332326572 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100134 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2920d8bd8ba10119189c084bf0e443 SystemUUID:ec2920d8-bd8b-a101-1918-9c084bf0e443 BootID:576c8f76-a11f-44d4-bb1d-fce5ae7a6aea Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:af:aa:ae:a3:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:af:aa:ae:a3:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:c5:88:96:28:50 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 14:11:28.372161 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.372159    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 14:11:28.372263 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.372252    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 14:11:28.375510 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.375490    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 14:11:28.375635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.375514    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-209.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 14:11:28.376237 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.376226    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 14:11:28.376274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.376239    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 14:11:28.376274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.376247    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:11:28.377080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.377070    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:11:28.378032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.378017    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:11:28.378231 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.378221    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 14:11:28.380780 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380769    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:11:28.380894 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380885    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 14:11:28.380923 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380914    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 14:11:28.380950 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380927    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 14:11:28.380950 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380945    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 14:11:28.381005 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380960    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 10 14:11:28.381005 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.380972    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 14:11:28.381355 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.381337    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:11:28.381388 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.381370    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:11:28.383396 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.383378    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 14:11:28.385610 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.385597    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 14:11:28.385652 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.385626    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 14:11:28.386515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386503    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386519    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386526    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386532    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386541    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386548    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386561    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 14:11:28.386566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386567    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 14:11:28.386763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386574    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 14:11:28.386763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386586    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 14:11:28.386763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386598    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 14:11:28.386763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.386603    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 14:11:28.387987 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.387976    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 14:11:28.387987 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.387987    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 14:11:28.391572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.391551    2581 server.go:1268] "Started kubelet"
Sep 10 14:11:28.391674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.391626    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 14:11:28.391754 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.391659    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 14:11:28.391754 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.391681    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 14:11:28.392377 ip-10-0-131-209 systemd[1]: Started Kubernetes Kubelet.
Sep 10 14:11:28.396754 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.396735    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 14:11:28.396904 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.396729    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 14:11:28.396904 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.396786    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 14:11:28.397268 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.397146    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-209.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 14:11:28.397359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.397339    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 14:11:28.398481 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.398447    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 14:11:28.399602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.399587    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 14:11:28.399686 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.399602    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 14:11:28.399838 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.399815    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.400241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400226    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 14:11:28.400353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400226    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 14:11:28.400353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400353    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 14:11:28.400508 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400492    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 14:11:28.400508 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400500    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 14:11:28.400603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400561    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 14:11:28.400603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400575    2581 factory.go:55] Registering systemd factory
Sep 10 14:11:28.400603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400584    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 10 14:11:28.400989 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400972    2581 factory.go:153] Registering CRI-O factory
Sep 10 14:11:28.400989 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.400992    2581 factory.go:223] Registration of the crio container factory successfully
Sep 10 14:11:28.401113 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.401009    2581 factory.go:103] Registering Raw factory
Sep 10 14:11:28.401113 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.401022    2581 manager.go:1196] Started watching for new ooms in manager
Sep 10 14:11:28.401308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.401297    2581 manager.go:349] Starting recovery of all containers
Sep 10 14:11:28.401993 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.400968    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07e92bc899  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.391522457 +0000 UTC m=+0.353031970,LastTimestamp:2026-09-10 14:11:28.391522457 +0000 UTC m=+0.353031970,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.411006 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.410976    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 14:11:28.411086 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.410995    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 14:11:28.411421 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.411407    2581 manager.go:354] Recovery completed
Sep 10 14:11:28.417306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.417294    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.419121 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419104    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.419192 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419132    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.419192 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419145    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.419649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419633    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 14:11:28.419649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419644    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.419649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419648    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 14:11:28.419826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.419665    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:11:28.420392 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420377    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.420453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420406    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.420453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420418    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.420954 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420941    2581 policy_none.go:49] "None policy: Start"
Sep 10 14:11:28.421001 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420958    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 14:11:28.421001 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.420970    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 14:11:28.422065 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.422054    2581 policy_none.go:47] "Start"
Sep 10 14:11:28.427424 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.427323    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead0ded2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419118802 +0000 UTC m=+0.380628312,LastTimestamp:2026-09-10 14:11:28.419118802 +0000 UTC m=+0.380628312,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.435547 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.435463    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead12f67  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419139431 +0000 UTC m=+0.380648949,LastTimestamp:2026-09-10 14:11:28.419139431 +0000 UTC m=+0.380648949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.447782 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.447708    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead15760  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419149664 +0000 UTC m=+0.380659173,LastTimestamp:2026-09-10 14:11:28.419149664 +0000 UTC m=+0.380659173,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457213    2581 manager.go:337] "Starting Device Plugin manager"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.457241    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457249    2581 server.go:87] "Starting device plugin registration server"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457471    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457483    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457561    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457641    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.457651    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.458304    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.458355    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.463024 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.461741    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-209.ec2.internal.18d3fb07ead0ded2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead0ded2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419118802 +0000 UTC m=+0.380628312,LastTimestamp:2026-09-10 14:11:28.420391876 +0000 UTC m=+0.381901386,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.469285 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.469229    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-209.ec2.internal.18d3fb07ead12f67\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead12f67  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419139431 +0000 UTC m=+0.380648949,LastTimestamp:2026-09-10 14:11:28.420411323 +0000 UTC m=+0.381920833,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.481896 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.481825    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-209.ec2.internal.18d3fb07ead15760\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ead15760  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-209.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.419149664 +0000 UTC m=+0.380659173,LastTimestamp:2026-09-10 14:11:28.420423895 +0000 UTC m=+0.381933405,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.497357 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.497296    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-209.ec2.internal.18d3fb07ed37cd59  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-209.ec2.internal,UID:ip-10-0-131-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-131-209.ec2.internal,},FirstTimestamp:2026-09-10 14:11:28.459418969 +0000 UTC m=+0.420928464,LastTimestamp:2026-09-10 14:11:28.459418969 +0000 UTC m=+0.420928464,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-209.ec2.internal,}"
Sep 10 14:11:28.523563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.523536    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 14:11:28.524674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.524653    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 14:11:28.524784 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.524707    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 14:11:28.524784 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.524729    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 14:11:28.524885 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.524804    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 14:11:28.532452 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.532428    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 10 14:11:28.540863 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.540848    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z5thr"
Sep 10 14:11:28.548648 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.548633    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z5thr"
Sep 10 14:11:28.558068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.558051    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.558844 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.558822    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.558919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.558858    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.558919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.558872    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.558919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.558898    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.574632 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.574488    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.574632 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.574613    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-209.ec2.internal\": node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.594805 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.594784    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.624897 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.624876    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"]
Sep 10 14:11:28.624978 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.624930    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.625654 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.625638    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.625733 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.625668    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.625733 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.625678    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.626913 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.626899    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.627024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627012    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.627069 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627037    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.627542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627524    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.627542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627543    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.627643 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627528    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.627643 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627554    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.627643 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627574    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.627643 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.627585    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.628670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.628657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.628740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.628677    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:28.629280 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.629262    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:28.629365 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.629291    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:28.629365 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.629306    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:28.650229 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.650212    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-209.ec2.internal\" not found" node="ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.654472 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.654457    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-209.ec2.internal\" not found" node="ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.695166 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.695141    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.702499 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.702484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be1a10a3d481e4ebb21a9b9822808141-config\") pod \"kube-apiserver-proxy-ip-10-0-131-209.ec2.internal\" (UID: \"be1a10a3d481e4ebb21a9b9822808141\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.702616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.702506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.702616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.702522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.795723 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.795686    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.803043 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.803120 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.803120 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.803200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be1a10a3d481e4ebb21a9b9822808141-config\") pod \"kube-apiserver-proxy-ip-10-0-131-209.ec2.internal\" (UID: \"be1a10a3d481e4ebb21a9b9822808141\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.803200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57d37a31eb133eddff6a63d7e7b7698a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal\" (UID: \"57d37a31eb133eddff6a63d7e7b7698a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.803200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.803150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be1a10a3d481e4ebb21a9b9822808141-config\") pod \"kube-apiserver-proxy-ip-10-0-131-209.ec2.internal\" (UID: \"be1a10a3d481e4ebb21a9b9822808141\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.896490 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.896428    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:28.952902 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.952876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.957665 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:28.957649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:28.997128 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:28.997104    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.097674 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.097652    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.198193 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.198133    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.301191 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.301156    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.340661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.340638    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 14:11:29.402256 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.402237    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.445999 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.445975    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:29.447045 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.447028    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:29.447117 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.447066    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:29.447117 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.447083    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:29.472918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.472867    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:29.502773 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.502753    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.551531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.551489    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:06:28 +0000 UTC" deadline="2028-04-26 22:15:39.611830926 +0000 UTC"
Sep 10 14:11:29.551531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.551525    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14264h4m10.06030937s"
Sep 10 14:11:29.603473 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.603447    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.608419 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.608403    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:29.704336 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:29.704313    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-209.ec2.internal\" not found"
Sep 10 14:11:29.742620 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.742554    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:29.800085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.800064    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:29.804218 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:29.804187    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1a10a3d481e4ebb21a9b9822808141.slice/crio-364d038360a975e2abb2c09f4952d35a6437b895522d451e0bfb3a74b0cd33aa WatchSource:0}: Error finding container 364d038360a975e2abb2c09f4952d35a6437b895522d451e0bfb3a74b0cd33aa: Status 404 returned error can't find the container with id 364d038360a975e2abb2c09f4952d35a6437b895522d451e0bfb3a74b0cd33aa
Sep 10 14:11:29.804654 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:29.804633    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d37a31eb133eddff6a63d7e7b7698a.slice/crio-dcb190d1e2bf5241239154f451f13402658f12121291c847cef1877b32a30372 WatchSource:0}: Error finding container dcb190d1e2bf5241239154f451f13402658f12121291c847cef1877b32a30372: Status 404 returned error can't find the container with id dcb190d1e2bf5241239154f451f13402658f12121291c847cef1877b32a30372
Sep 10 14:11:29.810293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.810269    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:11:29.815591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.814000    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:11:29.815591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.814816    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal"
Sep 10 14:11:29.821657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.821637    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:11:29.923409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:29.923380    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:30.384291 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.384261    2581 apiserver.go:52] "Watching apiserver"
Sep 10 14:11:30.394157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.394133    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 14:11:30.394495 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.394474    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal","openshift-network-diagnostics/network-check-target-z87gj","openshift-network-operator/iptables-alerter-42v4s","openshift-ovn-kubernetes/ovnkube-node-jnxx6","kube-system/global-pull-secret-syncer-8rqck","kube-system/konnectivity-agent-vp8wd","kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4","openshift-dns/node-resolver-96r9b","openshift-multus/multus-7fdm7","openshift-multus/multus-additional-cni-plugins-xkbrc","openshift-multus/network-metrics-daemon-6ghsl","openshift-cluster-node-tuning-operator/tuned-s22pt","openshift-image-registry/node-ca-wmknf"]
Sep 10 14:11:30.396813 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.396794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.398163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.398149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.399687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.399671    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.399753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.399712    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6l56n\""
Sep 10 14:11:30.399835 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.399819    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 14:11:30.399942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.399926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.400194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.400102    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.400755 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.400730    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.401127 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.401109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.401127 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.401122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fltsx\""
Sep 10 14:11:30.401565 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.401544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.402397 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.402378    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 14:11:30.402493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.402381    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.402767 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.402752    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.402850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.402771    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 14:11:30.403106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.403088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.403200 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.403176    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404319    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2rfm4\""
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404387    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404403    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.404516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404403    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5qvkw\""
Sep 10 14:11:30.404894 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.404637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.405818 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.405797    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 14:11:30.405916 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.405816    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.406481 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.406461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.408391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.408371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.409083 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409057    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cg8qz\""
Sep 10 14:11:30.409185 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 14:11:30.409185 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409170    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 14:11:30.409311 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409295    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.409415 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409397    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6wc2\""
Sep 10 14:11:30.409467 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 14:11:30.409467 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.409932 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.409911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.411175 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.411158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.412152 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.411460    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412506    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-modprobe-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-lib-modules\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412712    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8vfw\" (UniqueName: \"kubernetes.io/projected/2760afbe-fffc-48b4-872e-0ca29afc35ae-kube-api-access-l8vfw\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-system-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-var-lib-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-etc-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-netd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-env-overrides\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412907    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysconfig\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-sys\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-host\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-bin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a767a03-df9e-4854-b713-d4d427d181da-tmp-dir\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.414476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-kubernetes\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-systemd\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-run\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-socket-dir-parent\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vclm\" (UniqueName: \"kubernetes.io/projected/2a767a03-df9e-4854-b713-d4d427d181da-kube-api-access-9vclm\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cni-binary-copy\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-daemon-config\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-kubelet\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-netns\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-node-log\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.412861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cm9gk\""
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.413985    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-log-socket\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-var-lib-kubelet\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-tuned\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.415406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-k8s-cni-cncf-io\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-ovn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-config\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-script-lib\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-conf\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-netns\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bjd7k\""
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-kubelet\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-hostroot\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-conf-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-multus-certs\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc76m\" (UniqueName: \"kubernetes.io/projected/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-kube-api-access-cc76m\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a767a03-df9e-4854-b713-d4d427d181da-hosts-file\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-tmp\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.414999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cnibin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-multus\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-etc-kubernetes\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.416213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-systemd-units\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-slash\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-systemd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-bin\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovn-node-metrics-cert\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.415534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsnjn\" (UniqueName: \"kubernetes.io/projected/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-kube-api-access-bsnjn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.416580    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:30.416799 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.416667    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:30.418021 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.418004    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:11:30.418123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.418028    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.420923 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.420906    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 14:11:30.421031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.421016    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xpdmt\""
Sep 10 14:11:30.421112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.421094    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 14:11:30.421220 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.421206    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 14:11:30.445240 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.445223    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qwr44"
Sep 10 14:11:30.456589 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.456573    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qwr44"
Sep 10 14:11:30.501255 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.501234    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 14:11:30.515955 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.515930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a767a03-df9e-4854-b713-d4d427d181da-hosts-file\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.515955 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.515955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cnibin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.515972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-multus\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.515989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-bin\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-registration-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cnibin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-multus\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-bin\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a767a03-df9e-4854-b713-d4d427d181da-hosts-file\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e6dbcd2-d378-4917-a184-0dd655be1320-serviceca\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovn-node-metrics-cert\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsnjn\" (UniqueName: \"kubernetes.io/projected/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-kube-api-access-bsnjn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af45afff-ba25-4cfb-82dd-caabe513b30a-konnectivity-ca\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e6dbcd2-d378-4917-a184-0dd655be1320-host\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-modprobe-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-lib-modules\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-modprobe-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516414    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-system-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-lib-modules\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhvbv\" (UniqueName: \"kubernetes.io/projected/303de911-209c-41d2-bf01-3f7a8666ef57-kube-api-access-lhvbv\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-sys\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-system-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09632ea2-992c-4bad-8473-9b8204b42ce0-host-slash\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-sys\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-system-cni-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-netd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-env-overrides\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.516753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516743    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-cni-netd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a767a03-df9e-4854-b713-d4d427d181da-tmp-dir\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-kubernetes\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-run\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vclm\" (UniqueName: \"kubernetes.io/projected/2a767a03-df9e-4854-b713-d4d427d181da-kube-api-access-9vclm\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-daemon-config\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-socket-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.516996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cnibin\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-d\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-run\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86dh2\" (UniqueName: \"kubernetes.io/projected/1e6dbcd2-d378-4917-a184-0dd655be1320-kube-api-access-86dh2\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-ovn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-kubernetes\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-config\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-ovn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517296    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 14:11:30.517506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-script-lib\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-var-lib-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-etc-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-conf\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-kubelet\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-env-overrides\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-multus-certs\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-slash\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-etc-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-systemd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-config\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-var-lib-openvswitch\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-kubelet\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-multus-certs\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-slash\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.518357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-run-systemd\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-tmp\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysctl-conf\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-etc-kubernetes\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-etc-kubernetes\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-systemd-units\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.517978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-device-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-systemd-units\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-daemon-config\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a767a03-df9e-4854-b713-d4d427d181da-tmp-dir\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovnkube-script-lib\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09632ea2-992c-4bad-8473-9b8204b42ce0-iptables-alerter-script\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwrp4\" (UniqueName: \"kubernetes.io/projected/858c0a41-f465-4cac-b5e5-8273e9bac7b3-kube-api-access-dwrp4\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-kubelet-config\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518213    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8vfw\" (UniqueName: \"kubernetes.io/projected/2760afbe-fffc-48b4-872e-0ca29afc35ae-kube-api-access-l8vfw\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.519019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysconfig\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-host\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-bin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97tv\" (UniqueName: \"kubernetes.io/projected/09632ea2-992c-4bad-8473-9b8204b42ce0-kube-api-access-w97tv\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-systemd\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-sysconfig\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-var-lib-cni-bin\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-host\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-systemd\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-socket-dir-parent\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.518999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-kubelet\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-socket-dir-parent\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-kubelet\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq2t6\" (UniqueName: \"kubernetes.io/projected/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-kube-api-access-nq2t6\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-ovn-kubernetes\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cni-binary-copy\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.519698 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-netns\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-node-log\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-log-socket\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-dbus\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-etc-selinux\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-host-run-netns\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-log-socket\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-var-lib-kubelet\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-tuned\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2760afbe-fffc-48b4-872e-0ca29afc35ae-var-lib-kubelet\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-node-log\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-k8s-cni-cncf-io\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-sys-fs\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-k8s-cni-cncf-io\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af45afff-ba25-4cfb-82dd-caabe513b30a-agent-certs\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-netns\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-hostroot\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-host-run-netns\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-hostroot\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-conf-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-conf-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc76m\" (UniqueName: \"kubernetes.io/projected/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-kube-api-access-cc76m\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-multus-cni-dir\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.520857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.519705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-cni-binary-copy\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.521103 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.520956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-ovn-node-metrics-cert\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.521103 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.520990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-tmp\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.521455 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.521440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2760afbe-fffc-48b4-872e-0ca29afc35ae-etc-tuned\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.525094 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.525073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsnjn\" (UniqueName: \"kubernetes.io/projected/a2f2bcda-a926-40e6-a7c2-73ef65ed0602-kube-api-access-bsnjn\") pod \"ovnkube-node-jnxx6\" (UID: \"a2f2bcda-a926-40e6-a7c2-73ef65ed0602\") " pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.529024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.528978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal" event={"ID":"57d37a31eb133eddff6a63d7e7b7698a","Type":"ContainerStarted","Data":"dcb190d1e2bf5241239154f451f13402658f12121291c847cef1877b32a30372"}
Sep 10 14:11:30.530760 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.530737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vclm\" (UniqueName: \"kubernetes.io/projected/2a767a03-df9e-4854-b713-d4d427d181da-kube-api-access-9vclm\") pod \"node-resolver-96r9b\" (UID: \"2a767a03-df9e-4854-b713-d4d427d181da\") " pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.530955 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.530935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal" event={"ID":"be1a10a3d481e4ebb21a9b9822808141","Type":"ContainerStarted","Data":"364d038360a975e2abb2c09f4952d35a6437b895522d451e0bfb3a74b0cd33aa"}
Sep 10 14:11:30.531075 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.531059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc76m\" (UniqueName: \"kubernetes.io/projected/64bf7fd1-d479-4272-a4e9-d9e5a950d89e-kube-api-access-cc76m\") pod \"multus-7fdm7\" (UID: \"64bf7fd1-d479-4272-a4e9-d9e5a950d89e\") " pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.531513 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.531496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8vfw\" (UniqueName: \"kubernetes.io/projected/2760afbe-fffc-48b4-872e-0ca29afc35ae-kube-api-access-l8vfw\") pod \"tuned-s22pt\" (UID: \"2760afbe-fffc-48b4-872e-0ca29afc35ae\") " pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.619885 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.619852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97tv\" (UniqueName: \"kubernetes.io/projected/09632ea2-992c-4bad-8473-9b8204b42ce0-kube-api-access-w97tv\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.619885 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.619884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq2t6\" (UniqueName: \"kubernetes.io/projected/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-kube-api-access-nq2t6\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.620067 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.619905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-dbus\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.620067 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.619982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-dbus\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.620067 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-etc-selinux\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-sys-fs\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af45afff-ba25-4cfb-82dd-caabe513b30a-agent-certs\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-etc-selinux\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-registration-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e6dbcd2-d378-4917-a184-0dd655be1320-serviceca\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-sys-fs\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af45afff-ba25-4cfb-82dd-caabe513b30a-konnectivity-ca\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.620225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e6dbcd2-d378-4917-a184-0dd655be1320-host\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhvbv\" (UniqueName: \"kubernetes.io/projected/303de911-209c-41d2-bf01-3f7a8666ef57-kube-api-access-lhvbv\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09632ea2-992c-4bad-8473-9b8204b42ce0-host-slash\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-system-cni-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-registration-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e6dbcd2-d378-4917-a184-0dd655be1320-host\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09632ea2-992c-4bad-8473-9b8204b42ce0-host-slash\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.620469    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.620535    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:11:31.120513837 +0000 UTC m=+3.082023355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-system-cni-dir\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-socket-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-socket-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cnibin\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86dh2\" (UniqueName: \"kubernetes.io/projected/1e6dbcd2-d378-4917-a184-0dd655be1320-kube-api-access-86dh2\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.620724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cnibin\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.620660    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.620751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:11:31.120736576 +0000 UTC m=+3.082246084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e6dbcd2-d378-4917-a184-0dd655be1320-serviceca\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-device-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-device-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/303de911-209c-41d2-bf01-3f7a8666ef57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.620952    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af45afff-ba25-4cfb-82dd-caabe513b30a-konnectivity-ca\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09632ea2-992c-4bad-8473-9b8204b42ce0-iptables-alerter-script\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwrp4\" (UniqueName: \"kubernetes.io/projected/858c0a41-f465-4cac-b5e5-8273e9bac7b3-kube-api-access-dwrp4\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-kubelet-config\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.621493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-kubelet-config\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:30.622013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.622013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/09632ea2-992c-4bad-8473-9b8204b42ce0-iptables-alerter-script\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.622013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.621621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/858c0a41-f465-4cac-b5e5-8273e9bac7b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.622530 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.622512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af45afff-ba25-4cfb-82dd-caabe513b30a-agent-certs\") pod \"konnectivity-agent-vp8wd\" (UID: \"af45afff-ba25-4cfb-82dd-caabe513b30a\") " pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.631943 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.631916    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:30.631943 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.631944    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:30.632108 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.631958    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:30.632108 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:30.632023    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:11:31.132003547 +0000 UTC m=+3.093513053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:30.632980 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.632951    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97tv\" (UniqueName: \"kubernetes.io/projected/09632ea2-992c-4bad-8473-9b8204b42ce0-kube-api-access-w97tv\") pod \"iptables-alerter-42v4s\" (UID: \"09632ea2-992c-4bad-8473-9b8204b42ce0\") " pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.633319 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.633298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhvbv\" (UniqueName: \"kubernetes.io/projected/303de911-209c-41d2-bf01-3f7a8666ef57-kube-api-access-lhvbv\") pod \"aws-ebs-csi-driver-node-bczl4\" (UID: \"303de911-209c-41d2-bf01-3f7a8666ef57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.633376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.633362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86dh2\" (UniqueName: \"kubernetes.io/projected/1e6dbcd2-d378-4917-a184-0dd655be1320-kube-api-access-86dh2\") pod \"node-ca-wmknf\" (UID: \"1e6dbcd2-d378-4917-a184-0dd655be1320\") " pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.634200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.634180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwrp4\" (UniqueName: \"kubernetes.io/projected/858c0a41-f465-4cac-b5e5-8273e9bac7b3-kube-api-access-dwrp4\") pod \"multus-additional-cni-plugins-xkbrc\" (UID: \"858c0a41-f465-4cac-b5e5-8273e9bac7b3\") " pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.635003 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.634964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq2t6\" (UniqueName: \"kubernetes.io/projected/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-kube-api-access-nq2t6\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:30.708226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.708204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96r9b"
Sep 10 14:11:30.716673 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.716643    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a767a03_df9e_4854_b713_d4d427d181da.slice/crio-6473ec97186cdf6a7db708395d90e1504ea430742ed49b7cd7cfffca9146d746 WatchSource:0}: Error finding container 6473ec97186cdf6a7db708395d90e1504ea430742ed49b7cd7cfffca9146d746: Status 404 returned error can't find the container with id 6473ec97186cdf6a7db708395d90e1504ea430742ed49b7cd7cfffca9146d746
Sep 10 14:11:30.716788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.716699    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s22pt"
Sep 10 14:11:30.724071 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.724049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fdm7"
Sep 10 14:11:30.724363 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.724343    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2760afbe_fffc_48b4_872e_0ca29afc35ae.slice/crio-29632c8168c26c991eaf828429395015be6e8e4439153da845685084c18cc2f6 WatchSource:0}: Error finding container 29632c8168c26c991eaf828429395015be6e8e4439153da845685084c18cc2f6: Status 404 returned error can't find the container with id 29632c8168c26c991eaf828429395015be6e8e4439153da845685084c18cc2f6
Sep 10 14:11:30.728330 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.728310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:30.733012 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.732989    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bf7fd1_d479_4272_a4e9_d9e5a950d89e.slice/crio-8d5f98682be602904f2f67542021ebaf2bda8f01379372a1ba78854740f1e808 WatchSource:0}: Error finding container 8d5f98682be602904f2f67542021ebaf2bda8f01379372a1ba78854740f1e808: Status 404 returned error can't find the container with id 8d5f98682be602904f2f67542021ebaf2bda8f01379372a1ba78854740f1e808
Sep 10 14:11:30.733697 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.733674    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:30.737360 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.737338    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f2bcda_a926_40e6_a7c2_73ef65ed0602.slice/crio-4344860e60da4ffa92966e73ff1fe331edb9177f8d581d65a0926d8dc834fa52 WatchSource:0}: Error finding container 4344860e60da4ffa92966e73ff1fe331edb9177f8d581d65a0926d8dc834fa52: Status 404 returned error can't find the container with id 4344860e60da4ffa92966e73ff1fe331edb9177f8d581d65a0926d8dc834fa52
Sep 10 14:11:30.738821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.738688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4"
Sep 10 14:11:30.741452 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.741431    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf45afff_ba25_4cfb_82dd_caabe513b30a.slice/crio-7f5bc788165901920eb0baffafbf716a2548e0421e6ecf4bc2be361aaa253fea WatchSource:0}: Error finding container 7f5bc788165901920eb0baffafbf716a2548e0421e6ecf4bc2be361aaa253fea: Status 404 returned error can't find the container with id 7f5bc788165901920eb0baffafbf716a2548e0421e6ecf4bc2be361aaa253fea
Sep 10 14:11:30.745218 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.745198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-42v4s"
Sep 10 14:11:30.746108 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.746089    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303de911_209c_41d2_bf01_3f7a8666ef57.slice/crio-32a71611dc12ef0dcaa337505808c72b4e1a2f16ac22c1758aa7a020c662c00b WatchSource:0}: Error finding container 32a71611dc12ef0dcaa337505808c72b4e1a2f16ac22c1758aa7a020c662c00b: Status 404 returned error can't find the container with id 32a71611dc12ef0dcaa337505808c72b4e1a2f16ac22c1758aa7a020c662c00b
Sep 10 14:11:30.751362 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.751346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xkbrc"
Sep 10 14:11:30.752949 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.752932    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09632ea2_992c_4bad_8473_9b8204b42ce0.slice/crio-ce8500c5d93ce5a7d72e64cddd0bd2a2d151d43913ab81c1e0923d959bebb120 WatchSource:0}: Error finding container ce8500c5d93ce5a7d72e64cddd0bd2a2d151d43913ab81c1e0923d959bebb120: Status 404 returned error can't find the container with id ce8500c5d93ce5a7d72e64cddd0bd2a2d151d43913ab81c1e0923d959bebb120
Sep 10 14:11:30.755515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:30.755500    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmknf"
Sep 10 14:11:30.758872 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.758854    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858c0a41_f465_4cac_b5e5_8273e9bac7b3.slice/crio-74c89349e0305782cfda9138eef12a338b040193ab9eef6f1dcb982729ab05ad WatchSource:0}: Error finding container 74c89349e0305782cfda9138eef12a338b040193ab9eef6f1dcb982729ab05ad: Status 404 returned error can't find the container with id 74c89349e0305782cfda9138eef12a338b040193ab9eef6f1dcb982729ab05ad
Sep 10 14:11:30.763275 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:11:30.763256    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6dbcd2_d378_4917_a184_0dd655be1320.slice/crio-ff5cecf1230dc4fae99372d86723068ef4a843cad8e30041ed293281a371446b WatchSource:0}: Error finding container ff5cecf1230dc4fae99372d86723068ef4a843cad8e30041ed293281a371446b: Status 404 returned error can't find the container with id ff5cecf1230dc4fae99372d86723068ef4a843cad8e30041ed293281a371446b
Sep 10 14:11:31.125390 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.125303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:31.125390 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.125357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:31.125596 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.125479    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:31.125596 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.125545    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:11:32.125530319 +0000 UTC m=+4.087039822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:31.125596 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.125571    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:31.125754 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.125628    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:11:32.125609003 +0000 UTC m=+4.087118515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:31.226347 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.226304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:31.226542 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.226492    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:31.226542 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.226515    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:31.226542 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.226532    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:31.226701 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.226600    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:11:32.226580814 +0000 UTC m=+4.188090314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:31.459023 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.458933    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:06:30 +0000 UTC" deadline="2028-06-10 15:10:44.76506065 +0000 UTC"
Sep 10 14:11:31.459023 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.458976    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15336h59m13.306089231s"
Sep 10 14:11:31.526001 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.525969    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:31.526198 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.526107    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:31.526582 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.526557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:31.526706 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:31.526678    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:31.541803 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.541773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmknf" event={"ID":"1e6dbcd2-d378-4917-a184-0dd655be1320","Type":"ContainerStarted","Data":"ff5cecf1230dc4fae99372d86723068ef4a843cad8e30041ed293281a371446b"}
Sep 10 14:11:31.544637 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.544572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerStarted","Data":"74c89349e0305782cfda9138eef12a338b040193ab9eef6f1dcb982729ab05ad"}
Sep 10 14:11:31.548050 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.547979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4" event={"ID":"303de911-209c-41d2-bf01-3f7a8666ef57","Type":"ContainerStarted","Data":"32a71611dc12ef0dcaa337505808c72b4e1a2f16ac22c1758aa7a020c662c00b"}
Sep 10 14:11:31.551728 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.551699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp8wd" event={"ID":"af45afff-ba25-4cfb-82dd-caabe513b30a","Type":"ContainerStarted","Data":"7f5bc788165901920eb0baffafbf716a2548e0421e6ecf4bc2be361aaa253fea"}
Sep 10 14:11:31.554498 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.554470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"4344860e60da4ffa92966e73ff1fe331edb9177f8d581d65a0926d8dc834fa52"}
Sep 10 14:11:31.565941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.563097    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fdm7" event={"ID":"64bf7fd1-d479-4272-a4e9-d9e5a950d89e","Type":"ContainerStarted","Data":"8d5f98682be602904f2f67542021ebaf2bda8f01379372a1ba78854740f1e808"}
Sep 10 14:11:31.570072 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.570042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s22pt" event={"ID":"2760afbe-fffc-48b4-872e-0ca29afc35ae","Type":"ContainerStarted","Data":"29632c8168c26c991eaf828429395015be6e8e4439153da845685084c18cc2f6"}
Sep 10 14:11:31.572068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.572027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-42v4s" event={"ID":"09632ea2-992c-4bad-8473-9b8204b42ce0","Type":"ContainerStarted","Data":"ce8500c5d93ce5a7d72e64cddd0bd2a2d151d43913ab81c1e0923d959bebb120"}
Sep 10 14:11:31.573555 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:31.573508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96r9b" event={"ID":"2a767a03-df9e-4854-b713-d4d427d181da","Type":"ContainerStarted","Data":"6473ec97186cdf6a7db708395d90e1504ea430742ed49b7cd7cfffca9146d746"}
Sep 10 14:11:32.134249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.134191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:32.134249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.134253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:32.134514 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.134430    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:32.134514 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.134481    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:32.134631 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.134496    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:11:34.134477372 +0000 UTC m=+6.095986888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:32.134631 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.134575    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:11:34.134555313 +0000 UTC m=+6.096064818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:32.235190 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.235148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:32.235388 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.235333    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:32.235388 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.235355    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:32.235388 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.235369    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:32.235569 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.235430    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:11:34.235411009 +0000 UTC m=+6.196920523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:32.459463 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.459346    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:06:30 +0000 UTC" deadline="2028-04-06 09:20:12.485266592 +0000 UTC"
Sep 10 14:11:32.459463 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.459412    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13771h8m40.025858821s"
Sep 10 14:11:32.526782 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:32.526749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:32.526994 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:32.526932    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:33.525006 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:33.524965    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:33.525499 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:33.525111    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:33.525562 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:33.525497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:33.525634 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:33.525590    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:34.152416 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:34.152380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:34.152608 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:34.152435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:34.152608 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.152551    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:34.152608 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.152554    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:34.152814 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.152612    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:11:38.152593484 +0000 UTC m=+10.114102998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:34.152814 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.152633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:11:38.152622872 +0000 UTC m=+10.114132373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:34.253473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:34.253432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:34.253658 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.253629    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:34.253658 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.253650    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:34.253982 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.253665    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:34.253982 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.253722    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:11:38.253702362 +0000 UTC m=+10.215211881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:34.525861 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:34.525788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:34.526307 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:34.525915    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:35.525872 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:35.525836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:35.526306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:35.525836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:35.526306 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:35.525978    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:35.526306 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:35.526049    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:36.526850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:36.526794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:36.527276 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:36.526928    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:37.525763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:37.525673    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:37.525940 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:37.525809    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:37.525940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:37.525889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:37.526058 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:37.526026    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:38.185093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:38.185061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:38.185492 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:38.185103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:38.185492 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.185208    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:38.185492 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.185224    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:38.185492 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.185274    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:11:46.185257194 +0000 UTC m=+18.146766716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:38.185492 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.185293    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:11:46.185285048 +0000 UTC m=+18.146794547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:38.285988 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:38.285947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:38.286160 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.286129    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:38.286160 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.286152    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:38.286268 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.286167    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:38.286268 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.286229    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:11:46.286210713 +0000 UTC m=+18.247720223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:38.526937 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:38.526417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:38.526937 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:38.526523    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:39.525194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:39.525091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:39.525194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:39.525092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:39.525661 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:39.525235    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:39.525661 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:39.525290    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:40.525785 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:40.525750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:40.526182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:40.525893    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:41.525736 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:41.525546    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:41.525876 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:41.525542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:41.525876 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:41.525810    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:41.525876 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:41.525859    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:42.525304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:42.525270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:42.525472 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:42.525403    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:43.525537 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:43.525508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:43.525958 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:43.525603    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:43.525958 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:43.525684    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:43.525958 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:43.525820    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:44.525984 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:44.525952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:44.526428 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:44.526063    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:45.525096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:45.525062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:45.525304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:45.525070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:45.525304 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:45.525162    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:45.525304 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:45.525283    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:46.239123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:46.239082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:46.239597 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:46.239182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:46.239597 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.239187    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:46.239597 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.239249    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:12:02.239230443 +0000 UTC m=+34.200739960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:11:46.239597 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.239258    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:46.239597 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.239290    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:12:02.239279794 +0000 UTC m=+34.200789309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:11:46.340252 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:46.340214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:46.340428 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.340380    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:11:46.340428 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.340402    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:11:46.340428 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.340414    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:46.340584 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.340479    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:02.340461563 +0000 UTC m=+34.301971079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:11:46.525493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:46.525416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:46.525649 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:46.525546    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:47.525078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:47.525040    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:47.525536 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:47.525170    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:47.525536 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:47.525236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:47.525536 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:47.525345    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:48.526540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:48.526510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:48.526962 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:48.526598    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:49.525494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.525203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:49.525592 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.525236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:49.525643 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:49.525584    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:49.525732 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:49.525667    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:49.608812 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.608781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal" event={"ID":"be1a10a3d481e4ebb21a9b9822808141","Type":"ContainerStarted","Data":"55d2cef4281b84c316e3c813ddbadb447a50a58a5c4422a37b723995b0877da4"}
Sep 10 14:11:49.610265 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.610237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmknf" event={"ID":"1e6dbcd2-d378-4917-a184-0dd655be1320","Type":"ContainerStarted","Data":"29f3f420d2fe58e54de9ab9fcc130a6062cb8e6a6a2616d67a195e7ff158df3f"}
Sep 10 14:11:49.611586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.611564    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="207fd5e0f95b3c080dff96ba9f983b94c2f988e2c622b7f58900bf246bcf3952" exitCode=0
Sep 10 14:11:49.611677 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.611631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"207fd5e0f95b3c080dff96ba9f983b94c2f988e2c622b7f58900bf246bcf3952"}
Sep 10 14:11:49.613211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.613191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4" event={"ID":"303de911-209c-41d2-bf01-3f7a8666ef57","Type":"ContainerStarted","Data":"a620b754f1b04df49599fab31005206b8c8c244bbb7b6e7582e0654914bf9ee7"}
Sep 10 14:11:49.614607 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.614585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp8wd" event={"ID":"af45afff-ba25-4cfb-82dd-caabe513b30a","Type":"ContainerStarted","Data":"3ee09ea74eff1f355a1a243bd664e48ff547dfa088dc4f79beccf80901550a4d"}
Sep 10 14:11:49.617530 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.617505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:11:49.617859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.617835    2581 generic.go:356] "Generic (PLEG): container finished" podID="a2f2bcda-a926-40e6-a7c2-73ef65ed0602" containerID="56759d17b27c687882b58ac0d49b541d02b3886bc1c38df8abe53fcc8294b3a4" exitCode=1
Sep 10 14:11:49.617941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.617899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"ee8422ed8a86964db32644ef662e8112e318cd3073390a2f700c5b7afb4ca5c2"}
Sep 10 14:11:49.617991 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.617955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"8b7bac58de2da713db6b60f157a181ec86b4a389b5781481350d05dfda4bb26a"}
Sep 10 14:11:49.618029 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.618002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"9ea6dbfbdee36445df9b5447e0e2c1b2fc0068854a5d86a546edf842c3e9b30e"}
Sep 10 14:11:49.618059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.618043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"b432dc6fbc0c0c1d35a76b1b1472e141dc1b6ca14e7d94bd9d94bc838b0ee869"}
Sep 10 14:11:49.618104 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.618091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerDied","Data":"56759d17b27c687882b58ac0d49b541d02b3886bc1c38df8abe53fcc8294b3a4"}
Sep 10 14:11:49.618154 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.618139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"ad68cee5e602f92b4db5e110a216e01a7a155cf10697e1fc4b7e4f0f9ca4a15e"}
Sep 10 14:11:49.619184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.619166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fdm7" event={"ID":"64bf7fd1-d479-4272-a4e9-d9e5a950d89e","Type":"ContainerStarted","Data":"914c9ab19cac9fc7ddeb253d862f03c880a74290dee7d8211b7aa0da1386cec3"}
Sep 10 14:11:49.620463 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.620440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s22pt" event={"ID":"2760afbe-fffc-48b4-872e-0ca29afc35ae","Type":"ContainerStarted","Data":"37d04608a006f0faf48149fbb9e02ecd3ff8f21ff16df240a80f72ed9742429d"}
Sep 10 14:11:49.621808 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.621790    2581 generic.go:356] "Generic (PLEG): container finished" podID="57d37a31eb133eddff6a63d7e7b7698a" containerID="25e1e77ded34a5b0cb2c2569d1d650b1636ecb782015812e2625c2fe1f056511" exitCode=0
Sep 10 14:11:49.621930 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.621851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal" event={"ID":"57d37a31eb133eddff6a63d7e7b7698a","Type":"ContainerDied","Data":"25e1e77ded34a5b0cb2c2569d1d650b1636ecb782015812e2625c2fe1f056511"}
Sep 10 14:11:49.622066 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.622026    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-209.ec2.internal" podStartSLOduration=19.622010413 podStartE2EDuration="19.622010413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.621571276 +0000 UTC m=+21.583080794" watchObservedRunningTime="2026-09-10 14:11:49.622010413 +0000 UTC m=+21.583519932"
Sep 10 14:11:49.623195 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.623179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96r9b" event={"ID":"2a767a03-df9e-4854-b713-d4d427d181da","Type":"ContainerStarted","Data":"fb63d6c8724da0966127f3adfb75305109eae86d08e7ef8aa029e8ac265f1542"}
Sep 10 14:11:49.635902 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.635871    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s22pt" podStartSLOduration=3.616824929 podStartE2EDuration="21.63586004s" totalImagesPullingTime="18.019035111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.635418558 +0000 UTC m=+21.596928098" watchObservedRunningTime="2026-09-10 14:11:49.63586004 +0000 UTC m=+21.597369557"
Sep 10 14:11:49.651546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.651512    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7fdm7" podStartSLOduration=2.618523464 podStartE2EDuration="20.651502783s" totalImagesPullingTime="18.032979319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.651167757 +0000 UTC m=+21.612677298" watchObservedRunningTime="2026-09-10 14:11:49.651502783 +0000 UTC m=+21.613012300"
Sep 10 14:11:49.699532 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.699497    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wmknf" podStartSLOduration=2.730713038 podStartE2EDuration="20.699486508s" totalImagesPullingTime="17.96877347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.699222997 +0000 UTC m=+21.660732515" watchObservedRunningTime="2026-09-10 14:11:49.699486508 +0000 UTC m=+21.660996026"
Sep 10 14:11:49.772567 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:49.772533    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vp8wd" podStartSLOduration=9.875086999 podStartE2EDuration="20.772523964s" totalImagesPullingTime="10.897436965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.733724734 +0000 UTC m=+21.695234242" watchObservedRunningTime="2026-09-10 14:11:49.772523964 +0000 UTC m=+21.734033481"
Sep 10 14:11:50.525076 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:50.524994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:50.525328 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:50.525112    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:50.626730 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:50.626668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-42v4s" event={"ID":"09632ea2-992c-4bad-8473-9b8204b42ce0","Type":"ContainerStarted","Data":"b514b7222b12e665660fc562f117134027ae3f72485c2eac19c31c9557026a46"}
Sep 10 14:11:50.641518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:50.641471    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-42v4s" podStartSLOduration=3.663953263 podStartE2EDuration="21.641455454s" totalImagesPullingTime="17.977502191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:50.641165757 +0000 UTC m=+22.602675276" watchObservedRunningTime="2026-09-10 14:11:50.641455454 +0000 UTC m=+22.602964971"
Sep 10 14:11:50.641995 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:50.641951    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-96r9b" podStartSLOduration=4.611534503 podStartE2EDuration="22.641941577s" totalImagesPullingTime="18.030407074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:49.772293231 +0000 UTC m=+21.733802749" watchObservedRunningTime="2026-09-10 14:11:50.641941577 +0000 UTC m=+22.603451096"
Sep 10 14:11:51.421811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.421757    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:51.422511 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.422487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:51.525285 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.525259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:51.525398 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:51.525379    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:51.525811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.525794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:51.525933 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:51.525899    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:51.595901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.595877    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 14:11:51.633662 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.633640    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:11:51.634358 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.634152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"7dbde2e9975866689b0bc4051c3817f239f0ba2f6d8468a73f43fc93d2912f4b"}
Sep 10 14:11:51.636553 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.636525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal" event={"ID":"57d37a31eb133eddff6a63d7e7b7698a","Type":"ContainerStarted","Data":"d951100cfdba8ed0b975c08d90bdfb2d8e3b24f323b29ae442d05b0e3d839d68"}
Sep 10 14:11:51.638759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.638735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4" event={"ID":"303de911-209c-41d2-bf01-3f7a8666ef57","Type":"ContainerStarted","Data":"99214b5017b24f74eaf5a0aea54159cd4cee2fac709743cf1723af920d04bb30"}
Sep 10 14:11:51.639060 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.639041    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:51.639390 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.639372    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vp8wd"
Sep 10 14:11:51.653377 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:51.653334    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-209.ec2.internal" podStartSLOduration=21.653318009 podStartE2EDuration="21.653318009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:51.652922298 +0000 UTC m=+23.614431817" watchObservedRunningTime="2026-09-10 14:11:51.653318009 +0000 UTC m=+23.614827527"
Sep 10 14:11:52.470877 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:52.470782    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:11:51.595897795Z","UUID":"11a1d998-2cb4-4607-bab5-9b3550eea16f","Handler":null,"Name":"","Endpoint":""}
Sep 10 14:11:52.472306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:52.472285    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 14:11:52.472306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:52.472315    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 14:11:52.525191 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:52.525166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:52.525303 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:52.525285    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:53.525891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:53.525685    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:53.526279 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:53.525723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:53.526279 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:53.526015    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:53.526279 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:53.526032    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:54.525407 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:54.525372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:54.525618 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:54.525502    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:55.525637 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.525557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:55.526008 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.525559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:55.526008 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:55.525651    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:55.526008 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:55.525751    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:55.649619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.649592    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="b7e0ae62e17566375bd9f6a1a063a71e856ceffded24b8a71132aa46978dfd28" exitCode=0
Sep 10 14:11:55.649783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.649663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"b7e0ae62e17566375bd9f6a1a063a71e856ceffded24b8a71132aa46978dfd28"}
Sep 10 14:11:55.653952 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.653923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4" event={"ID":"303de911-209c-41d2-bf01-3f7a8666ef57","Type":"ContainerStarted","Data":"e4ee1d9d21f4c85eff3e20e56853924853837836aee331a7ab8f214d0c6e4cd1"}
Sep 10 14:11:55.659656 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.659635    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:11:55.660055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.660034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"3a98cb01f95cc9cda9e3354a65216d899e6ce69ebaf26dc78909f043c7e40ec0"}
Sep 10 14:11:55.660250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.660232    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:55.660314 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.660293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:55.660438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.660423    2581 scope.go:117] "RemoveContainer" containerID="56759d17b27c687882b58ac0d49b541d02b3886bc1c38df8abe53fcc8294b3a4"
Sep 10 14:11:55.675132 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.675115    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:55.697733 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:55.697677    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bczl4" podStartSLOduration=10.542026073 podStartE2EDuration="26.697663271s" totalImagesPullingTime="16.155637198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:55.697533047 +0000 UTC m=+27.659042564" watchObservedRunningTime="2026-09-10 14:11:55.697663271 +0000 UTC m=+27.659172789"
Sep 10 14:11:56.525332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.525309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:56.525505 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:56.525478    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:56.665480 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.665404    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="de3e74fcd0a291eb1797bcacc47dd82583b2147da1006df21c7981934a53d990" exitCode=0
Sep 10 14:11:56.666021 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.665507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"de3e74fcd0a291eb1797bcacc47dd82583b2147da1006df21c7981934a53d990"}
Sep 10 14:11:56.669811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.669792    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:11:56.670193 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.670141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" event={"ID":"a2f2bcda-a926-40e6-a7c2-73ef65ed0602","Type":"ContainerStarted","Data":"4f0dbd9c12f4407f7e9cebdef9755414572542d690e325ee8bb080494d44884d"}
Sep 10 14:11:56.670472 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.670450    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:56.684286 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.684269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:11:56.712888 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.712846    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6" podStartSLOduration=9.48725983 podStartE2EDuration="27.712832391s" totalImagesPullingTime="18.225572561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:11:56.712646313 +0000 UTC m=+28.674155844" watchObservedRunningTime="2026-09-10 14:11:56.712832391 +0000 UTC m=+28.674341908"
Sep 10 14:11:56.895912 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.895879    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8rqck"]
Sep 10 14:11:56.896058 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.896011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:56.896156 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:56.896131    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:56.898982 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.898962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z87gj"]
Sep 10 14:11:56.899083 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.899044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:56.899122 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:56.899104    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:56.901402 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.901376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6ghsl"]
Sep 10 14:11:56.901500 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:56.901487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:56.901597 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:56.901576    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:58.526807 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:58.526563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:11:58.527274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:58.526648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:11:58.527274 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:58.526815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:11:58.527274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:58.526670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:11:58.527274 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:58.526889    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:11:58.527274 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:11:58.526987    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:11:58.677438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:58.677407    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="60297edf427f4e5792816350ee4a8286a43b198b73bd9875450582ef5010d8e6" exitCode=0
Sep 10 14:11:58.677588 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:11:58.677481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"60297edf427f4e5792816350ee4a8286a43b198b73bd9875450582ef5010d8e6"}
Sep 10 14:12:00.526055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:00.526017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:00.526055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:00.526046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:12:00.526461 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:00.526123    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8rqck" podUID="a4090977-ff09-4a16-bed8-f4d8ba6a4efa"
Sep 10 14:12:00.526461 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:00.526176    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:12:00.526461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:00.526248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:00.526461 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:00.526315    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z87gj" podUID="b9197f42-6390-4968-82ba-a11710dc2850"
Sep 10 14:12:01.794506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.794441    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-209.ec2.internal" event="NodeReady"
Sep 10 14:12:01.794974 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.794572    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 14:12:01.834123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.834089    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:12:01.870650 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.870620    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2hd5g"]
Sep 10 14:12:01.870800 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.870774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.873655 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.873631    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 14:12:01.873655 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.873647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 14:12:01.873982 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.873961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wzphp\""
Sep 10 14:12:01.873982 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.873977    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 14:12:01.878457 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.878430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:12:01.878559 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.878532    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hd5g"]
Sep 10 14:12:01.878559 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.878535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:01.881934 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.881559    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 14:12:01.881934 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.881671    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwswr\""
Sep 10 14:12:01.882127 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.882110    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 14:12:01.885795 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.885774    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 14:12:01.948953 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.948929    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sx4k4"]
Sep 10 14:12:01.957108 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:01.957220 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:01.957332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd5w6\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqxc\" (UniqueName: \"kubernetes.io/projected/68e8319c-15ef-4012-b590-a364beff74e7-kube-api-access-9hqxc\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:01.957618 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68e8319c-15ef-4012-b590-a364beff74e7-tmp-dir\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:01.957618 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:01.957681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.957640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68e8319c-15ef-4012-b590-a364beff74e7-config-volume\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:01.960129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.960112    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4qtdq\""
Sep 10 14:12:01.960198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.960173    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 14:12:01.960198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.960183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 14:12:01.960298 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.960283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 14:12:01.964161 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:01.964142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx4k4"]
Sep 10 14:12:02.039939 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.039892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vp8wd_af45afff-ba25-4cfb-82dd-caabe513b30a/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:02.058878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-cert\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.058878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68e8319c-15ef-4012-b590-a364beff74e7-tmp-dir\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68e8319c-15ef-4012-b590-a364beff74e7-config-volume\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvm7h\" (UniqueName: \"kubernetes.io/projected/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-kube-api-access-kvm7h\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.058997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.059337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059337 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.059175    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:02.059337 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.059255    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:02.55923334 +0000 UTC m=+34.520742851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:02.059539 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.059363    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:02.059539 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.059378    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:02.059539 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.059437    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:02.559419133 +0000 UTC m=+34.520928649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:02.059539 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd5w6\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059539 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqxc\" (UniqueName: \"kubernetes.io/projected/68e8319c-15ef-4012-b590-a364beff74e7-kube-api-access-9hqxc\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.059825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.059825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68e8319c-15ef-4012-b590-a364beff74e7-config-volume\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.060022 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.059985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.060091 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.060070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.060314 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.060296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68e8319c-15ef-4012-b590-a364beff74e7-tmp-dir\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.063602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.063572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.063731 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.063600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.068151 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.068130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd5w6\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.068230 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.068181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.068633 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.068614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqxc\" (UniqueName: \"kubernetes.io/projected/68e8319c-15ef-4012-b590-a364beff74e7-kube-api-access-9hqxc\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.147774 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.147714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vp8wd_af45afff-ba25-4cfb-82dd-caabe513b30a/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:02.160595 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.160565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvm7h\" (UniqueName: \"kubernetes.io/projected/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-kube-api-access-kvm7h\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.160763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.160686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-cert\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.163551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.163523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-cert\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.170892 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.170866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvm7h\" (UniqueName: \"kubernetes.io/projected/1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4-kube-api-access-kvm7h\") pod \"ingress-canary-sx4k4\" (UID: \"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4\") " pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.261256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.261225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:12:02.261407 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.261263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:02.261407 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.261381    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:02.261407 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.261383    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:02.261563 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.261438    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret podName:a4090977-ff09-4a16-bed8-f4d8ba6a4efa nodeName:}" failed. No retries permitted until 2026-09-10 14:12:34.261422641 +0000 UTC m=+66.222932142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret") pod "global-pull-secret-syncer-8rqck" (UID: "a4090977-ff09-4a16-bed8-f4d8ba6a4efa") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:02.261563 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.261458    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:12:34.261448564 +0000 UTC m=+66.222958060 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:02.266150 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.266132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx4k4"
Sep 10 14:12:02.361930 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.361853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:02.362060 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.361976    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:02.362060 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.361998    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:02.362060 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.362010    2581 projected.go:196] Error preparing data for projected volume kube-api-access-xh7sr for pod openshift-network-diagnostics/network-check-target-z87gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:02.362190 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.362072    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr podName:b9197f42-6390-4968-82ba-a11710dc2850 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:34.36206018 +0000 UTC m=+66.323569681 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xh7sr" (UniqueName: "kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr") pod "network-check-target-z87gj" (UID: "b9197f42-6390-4968-82ba-a11710dc2850") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:02.525790 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.525753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:02.525957 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.525761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:12:02.525957 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.525761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:02.529117 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529085    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:12:02.529117 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529111    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:12:02.529306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529147    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:12:02.529306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529201    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:12:02.529306 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4vfcf\""
Sep 10 14:12:02.529581 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.529308    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jzwbc\""
Sep 10 14:12:02.564615 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.564591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:02.564753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:02.564635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:02.564818 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.564759    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:02.564818 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.564770    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:02.564818 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.564786    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:02.564957 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.564833    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:03.564816291 +0000 UTC m=+35.526325791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:02.564957 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:02.564850    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:03.564840996 +0000 UTC m=+35.526350495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:03.573972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:03.574030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:03.574241    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:03.574280    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:03.574294    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:03.574380    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:05.574345382 +0000 UTC m=+37.535854895 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:03.576640 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:03.574401    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:05.574390167 +0000 UTC m=+37.535899666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:03.827756 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:03.827729    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx4k4"]
Sep 10 14:12:03.831439 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:12:03.831410    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c34ec40_6ca5_4fe5_8a19_ed1fdc1770f4.slice/crio-36041d94c6f847c186e0b1a3f4d935a2c1e47314db46dd6785f4dc0e1f04f3fb WatchSource:0}: Error finding container 36041d94c6f847c186e0b1a3f4d935a2c1e47314db46dd6785f4dc0e1f04f3fb: Status 404 returned error can't find the container with id 36041d94c6f847c186e0b1a3f4d935a2c1e47314db46dd6785f4dc0e1f04f3fb
Sep 10 14:12:04.694910 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:04.694875    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="245e1add0dd725629ffef4b5fc56f72f68347e1bcffc3852922d0632efea15b7" exitCode=0
Sep 10 14:12:04.695315 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:04.694959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"245e1add0dd725629ffef4b5fc56f72f68347e1bcffc3852922d0632efea15b7"}
Sep 10 14:12:04.696026 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:04.696005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx4k4" event={"ID":"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4","Type":"ContainerStarted","Data":"36041d94c6f847c186e0b1a3f4d935a2c1e47314db46dd6785f4dc0e1f04f3fb"}
Sep 10 14:12:05.590354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:05.590323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:05.590560 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:05.590363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:05.590560 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:05.590495    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:05.590560 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:05.590497    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:05.590719 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:05.590573    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:09.590554796 +0000 UTC m=+41.552064309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:05.590719 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:05.590508    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:05.590719 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:05.590636    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:09.590621001 +0000 UTC m=+41.552130502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:05.702491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:05.702460    2581 generic.go:356] "Generic (PLEG): container finished" podID="858c0a41-f465-4cac-b5e5-8273e9bac7b3" containerID="fedad0e967ff7139334d5eb3148f7d09a8c77d69171bc3c4ae2f38f813346c87" exitCode=0
Sep 10 14:12:05.703005 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:05.702504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerDied","Data":"fedad0e967ff7139334d5eb3148f7d09a8c77d69171bc3c4ae2f38f813346c87"}
Sep 10 14:12:06.709368 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:06.709279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" event={"ID":"858c0a41-f465-4cac-b5e5-8273e9bac7b3","Type":"ContainerStarted","Data":"4f47a838e89bf52099d94c5dc8b7567c46c80cdcfa2b271a3f4aca8e58d3ba03"}
Sep 10 14:12:06.732886 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:06.732839    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xkbrc" podStartSLOduration=7.60071722 podStartE2EDuration="37.732823657s" totalImagesPullingTime="30.132106437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:06.731299585 +0000 UTC m=+38.692809107" watchObservedRunningTime="2026-09-10 14:12:06.732823657 +0000 UTC m=+38.694333180"
Sep 10 14:12:07.713563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:07.713474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx4k4" event={"ID":"1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4","Type":"ContainerStarted","Data":"394c44229d200bf8a48d4110c4c89405b33c35f057cbf7404212c87d986826c9"}
Sep 10 14:12:07.731574 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:07.731509    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sx4k4" podStartSLOduration=3.292721172 podStartE2EDuration="6.731496677s" totalImagesPullingTime="3.438775505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:07.730990734 +0000 UTC m=+39.692500251" watchObservedRunningTime="2026-09-10 14:12:07.731496677 +0000 UTC m=+39.693006234"
Sep 10 14:12:09.620682 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:09.620649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:09.620682 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:09.620689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:09.621085 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:09.620791    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:09.621085 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:09.620792    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:09.621085 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:09.620852    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:17.620836036 +0000 UTC m=+49.582345545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:09.621085 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:09.620801    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:09.621085 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:09.620928    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:17.620916439 +0000 UTC m=+49.582425935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:17.675203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:17.675167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:17.675203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:17.675209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:17.675671 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:17.675315    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:17.675671 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:17.675317    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:17.675671 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:17.675393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:33.675377463 +0000 UTC m=+65.636886984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:17.675671 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:17.675328    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:17.675671 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:17.675464    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:12:33.675452246 +0000 UTC m=+65.636961743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:28.687258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:28.687226    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jnxx6"
Sep 10 14:12:33.682165 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:33.682134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:33.682176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:33.682275    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:33.682293    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:33.682304    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:33.682343    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:05.682326007 +0000 UTC m=+97.643835507 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:33.682571 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:33.682359    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:13:05.682352998 +0000 UTC m=+97.643862494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:12:34.284795 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.284764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:12:34.284795 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.284799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:34.288127 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.288108    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:12:34.288185 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.288130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:12:34.295622 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:34.295590    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:12:34.295685 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:12:34.295655    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:13:38.295641443 +0000 UTC m=+130.257150939 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : secret "metrics-daemon-secret" not found
Sep 10 14:12:34.298587 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.298562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4090977-ff09-4a16-bed8-f4d8ba6a4efa-original-pull-secret\") pod \"global-pull-secret-syncer-8rqck\" (UID: \"a4090977-ff09-4a16-bed8-f4d8ba6a4efa\") " pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:34.350128 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.350103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8rqck"
Sep 10 14:12:34.385711 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.385682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:34.388856 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.388841    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:12:34.399518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.399498    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:12:34.409343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.409324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7sr\" (UniqueName: \"kubernetes.io/projected/b9197f42-6390-4968-82ba-a11710dc2850-kube-api-access-xh7sr\") pod \"network-check-target-z87gj\" (UID: \"b9197f42-6390-4968-82ba-a11710dc2850\") " pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:34.640639 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.640581    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4vfcf\""
Sep 10 14:12:34.648051 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.648028    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:34.651002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.650981    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8rqck"]
Sep 10 14:12:34.654411 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:12:34.654386    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4090977_ff09_4a16_bed8_f4d8ba6a4efa.slice/crio-0d6a1b7a6d3846e5b61b069a8c2bbc01e59bfc6bd21364ef56a5cf44cabca6fa WatchSource:0}: Error finding container 0d6a1b7a6d3846e5b61b069a8c2bbc01e59bfc6bd21364ef56a5cf44cabca6fa: Status 404 returned error can't find the container with id 0d6a1b7a6d3846e5b61b069a8c2bbc01e59bfc6bd21364ef56a5cf44cabca6fa
Sep 10 14:12:34.774949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.774917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8rqck" event={"ID":"a4090977-ff09-4a16-bed8-f4d8ba6a4efa","Type":"ContainerStarted","Data":"0d6a1b7a6d3846e5b61b069a8c2bbc01e59bfc6bd21364ef56a5cf44cabca6fa"}
Sep 10 14:12:34.934311 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:34.934135    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z87gj"]
Sep 10 14:12:34.937302 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:12:34.937276    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9197f42_6390_4968_82ba_a11710dc2850.slice/crio-e2563da1413b3e63fcb53d485f7314e611b39e7b07c3f3ad2cea1fc00e3d9ecd WatchSource:0}: Error finding container e2563da1413b3e63fcb53d485f7314e611b39e7b07c3f3ad2cea1fc00e3d9ecd: Status 404 returned error can't find the container with id e2563da1413b3e63fcb53d485f7314e611b39e7b07c3f3ad2cea1fc00e3d9ecd
Sep 10 14:12:35.779908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:35.779869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z87gj" event={"ID":"b9197f42-6390-4968-82ba-a11710dc2850","Type":"ContainerStarted","Data":"e2563da1413b3e63fcb53d485f7314e611b39e7b07c3f3ad2cea1fc00e3d9ecd"}
Sep 10 14:12:39.790523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:39.790477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8rqck" event={"ID":"a4090977-ff09-4a16-bed8-f4d8ba6a4efa","Type":"ContainerStarted","Data":"0b6850003d341f1e7545128ea81e798743754225e198fb8b29ee471e82e7ceb7"}
Sep 10 14:12:39.791925 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:39.791902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z87gj" event={"ID":"b9197f42-6390-4968-82ba-a11710dc2850","Type":"ContainerStarted","Data":"0f17b6d9b54eeb09cf0db57951639d248d5f2efb091b2bb1c25960566165f461"}
Sep 10 14:12:39.792039 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:39.792028    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:12:39.809540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:39.809502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8rqck" podStartSLOduration=65.988581235 podStartE2EDuration="1m10.809489828s" totalImagesPullingTime="4.820908593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:39.808058141 +0000 UTC m=+71.769567658" watchObservedRunningTime="2026-09-10 14:12:39.809489828 +0000 UTC m=+71.770999346"
Sep 10 14:12:39.825263 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:12:39.825220    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z87gj" podStartSLOduration=66.293042418 podStartE2EDuration="1m10.825207986s" totalImagesPullingTime="4.532165568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:39.825032551 +0000 UTC m=+71.786542071" watchObservedRunningTime="2026-09-10 14:12:39.825207986 +0000 UTC m=+71.786717504"
Sep 10 14:13:05.696695 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:05.696584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:13:05.696695 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:05.696624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:13:05.697182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:05.696735    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:13:05.697182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:05.696791    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls podName:68e8319c-15ef-4012-b590-a364beff74e7 nodeName:}" failed. No retries permitted until 2026-09-10 14:14:09.696777261 +0000 UTC m=+161.658286761 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls") pod "dns-default-2hd5g" (UID: "68e8319c-15ef-4012-b590-a364beff74e7") : secret "dns-default-metrics-tls" not found
Sep 10 14:13:05.697182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:05.696738    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 14:13:05.697182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:05.696831    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77bd49b46f-xktx4: secret "image-registry-tls" not found
Sep 10 14:13:05.697182 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:05.696910    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls podName:a1b18c3c-0e1b-486e-a94b-46355674210c nodeName:}" failed. No retries permitted until 2026-09-10 14:14:09.696896924 +0000 UTC m=+161.658406426 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls") pod "image-registry-77bd49b46f-xktx4" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c") : secret "image-registry-tls" not found
Sep 10 14:13:10.796017 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:10.795988    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z87gj"
Sep 10 14:13:28.473428 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.473394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"]
Sep 10 14:13:28.475163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.475139    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.479647 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.479617    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-29wb5\""
Sep 10 14:13:28.482177 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.482157    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 14:13:28.482289 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.482244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 14:13:28.488941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.488923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 14:13:28.489198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.489183    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 14:13:28.514251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.514225    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"]
Sep 10 14:13:28.548051 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.548031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.548149 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.548061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.548149 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.548086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmtgp\" (UniqueName: \"kubernetes.io/projected/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-kube-api-access-dmtgp\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.570017 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.569996    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-22b5c"]
Sep 10 14:13:28.571778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.571764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.577946 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.577931    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 14:13:28.578803 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.578782    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 14:13:28.578855 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.578805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 14:13:28.581776 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.581760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-n24xz\""
Sep 10 14:13:28.581933 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.581916    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 14:13:28.589250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.589235    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 14:13:28.604393 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.604372    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-22b5c"]
Sep 10 14:13:28.648828 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.648909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-tmp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.648909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.648909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-snapshots\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.648909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.649041 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmtgp\" (UniqueName: \"kubernetes.io/projected/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-kube-api-access-dmtgp\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.649041 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:28.648970    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:28.649041 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.648987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n9pp\" (UniqueName: \"kubernetes.io/projected/1c283e68-f138-4300-8e04-bb7fb2727d30-kube-api-access-2n9pp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.649041 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:28.649021    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:29.149003239 +0000 UTC m=+121.110512750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:28.649196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.649039    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.649196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.649066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c283e68-f138-4300-8e04-bb7fb2727d30-serving-cert\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.649456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.649439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.657895 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.657876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmtgp\" (UniqueName: \"kubernetes.io/projected/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-kube-api-access-dmtgp\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:28.749681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n9pp\" (UniqueName: \"kubernetes.io/projected/1c283e68-f138-4300-8e04-bb7fb2727d30-kube-api-access-2n9pp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.749681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.749681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c283e68-f138-4300-8e04-bb7fb2727d30-serving-cert\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.749881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-tmp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.749881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-snapshots\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.749881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.749779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.750143 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.750122    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-tmp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.750239 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.750220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.750519 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.750496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c283e68-f138-4300-8e04-bb7fb2727d30-snapshots\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.750591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.750549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c283e68-f138-4300-8e04-bb7fb2727d30-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.751832 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.751812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c283e68-f138-4300-8e04-bb7fb2727d30-serving-cert\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.760106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.760080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n9pp\" (UniqueName: \"kubernetes.io/projected/1c283e68-f138-4300-8e04-bb7fb2727d30-kube-api-access-2n9pp\") pod \"insights-operator-76bcd6ccf5-22b5c\" (UID: \"1c283e68-f138-4300-8e04-bb7fb2727d30\") " pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:28.879940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:28.879905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c"
Sep 10 14:13:29.152754 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:29.152714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:29.152929 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:29.152854    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:29.152988 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:29.152936    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:30.152914328 +0000 UTC m=+122.114423829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:29.153568 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:29.153547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-22b5c"]
Sep 10 14:13:29.156332 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:13:29.156310    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c283e68_f138_4300_8e04_bb7fb2727d30.slice/crio-cd65102b3e0d64883d3a6c0c2d4ec6e427a3af7676015b64bdcc1cd804b5d292 WatchSource:0}: Error finding container cd65102b3e0d64883d3a6c0c2d4ec6e427a3af7676015b64bdcc1cd804b5d292: Status 404 returned error can't find the container with id cd65102b3e0d64883d3a6c0c2d4ec6e427a3af7676015b64bdcc1cd804b5d292
Sep 10 14:13:29.908550 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:29.908513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c" event={"ID":"1c283e68-f138-4300-8e04-bb7fb2727d30","Type":"ContainerStarted","Data":"cd65102b3e0d64883d3a6c0c2d4ec6e427a3af7676015b64bdcc1cd804b5d292"}
Sep 10 14:13:30.159419 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:30.159346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:30.159559 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:30.159464    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:30.159559 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:30.159516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:32.159504724 +0000 UTC m=+124.121014220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:31.915647 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:31.915601    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c" event={"ID":"1c283e68-f138-4300-8e04-bb7fb2727d30","Type":"ContainerStarted","Data":"cfcce862de28b4927f0429c88ffe77eb08c275c02f1a969b0490b43b04801a8b"}
Sep 10 14:13:32.176111 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:32.176011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:32.176252 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:32.176157    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:32.176252 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:32.176221    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.176205625 +0000 UTC m=+128.137715125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:33.143157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.143090    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c" podStartSLOduration=2.89118569 podStartE2EDuration="5.143064615s" totalImagesPullingTime="2.251878925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:31.93839834 +0000 UTC m=+123.899907859" watchObservedRunningTime="2026-09-10 14:13:33.143064615 +0000 UTC m=+125.104574133"
Sep 10 14:13:33.143653 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.143623    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xzzw5"]
Sep 10 14:13:33.146047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.146026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.148661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.148640    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2mhjl\""
Sep 10 14:13:33.148851 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.148839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 14:13:33.149814 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.149800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 14:13:33.157509 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.157489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xzzw5"]
Sep 10 14:13:33.282273 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.282245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g46mq\" (UniqueName: \"kubernetes.io/projected/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-api-access-g46mq\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.282398 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.282361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd51d5fd-2863-44ce-82fc-f1da8a53857b-data-volume\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.282438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.282398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.282438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.282420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd51d5fd-2863-44ce-82fc-f1da8a53857b-crio-socket\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.282516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.282443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.382995 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.382966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd51d5fd-2863-44ce-82fc-f1da8a53857b-data-volume\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.382995 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.382999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.383162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.383024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd51d5fd-2863-44ce-82fc-f1da8a53857b-crio-socket\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.383162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.383085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd51d5fd-2863-44ce-82fc-f1da8a53857b-crio-socket\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.383162 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.383120    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.383265 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.383167    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.383265 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.383189    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:33.883167527 +0000 UTC m=+125.844677037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.383265 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.383127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.383265 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.383221    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:33.883204407 +0000 UTC m=+125.844713910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.383419 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.383263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g46mq\" (UniqueName: \"kubernetes.io/projected/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-api-access-g46mq\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.383455 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.383429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd51d5fd-2863-44ce-82fc-f1da8a53857b-data-volume\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.391194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.391165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g46mq\" (UniqueName: \"kubernetes.io/projected/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-api-access-g46mq\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.887968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.887926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.887968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:33.887974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:33.888305 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.888095    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.888305 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.888119    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.888305 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.888152    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:34.888136404 +0000 UTC m=+126.849645920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.888305 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:33.888190    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:34.888170333 +0000 UTC m=+126.849679845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:34.896535 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:34.896500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:34.896950 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:34.896588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:34.896950 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:34.896630    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:34.896950 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:34.896669    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:34.896950 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:34.896688    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.896674917 +0000 UTC m=+128.858184413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:34.896950 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:34.896728    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.896713428 +0000 UTC m=+128.858222937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:35.718965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:35.718936    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96r9b_2a767a03-df9e-4854-b713-d4d427d181da/dns-node-resolver/0.log"
Sep 10 14:13:36.207075 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:36.207041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:36.207442 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.207184    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:36.207442 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.207244    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:44.207229704 +0000 UTC m=+136.168739206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:36.912891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:36.912855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:36.912891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:36.912894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:36.913145 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.912995    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:36.913145 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.913029    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:36.913145 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.913047    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:40.913033942 +0000 UTC m=+132.874543438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:36.913145 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:36.913089    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:40.913072983 +0000 UTC m=+132.874582499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:36.926553 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:36.926528    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmknf_1e6dbcd2-d378-4917-a184-0dd655be1320/node-ca/0.log"
Sep 10 14:13:37.529234 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:37.529205    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx4k4_1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4/serve-healthcheck-canary/0.log"
Sep 10 14:13:38.323232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:38.323200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:13:38.323396 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:38.323352    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:13:38.323438 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:38.323413    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs podName:b9c4139c-2dde-4cd3-ba84-c9bee977f4ea nodeName:}" failed. No retries permitted until 2026-09-10 14:15:40.323399392 +0000 UTC m=+252.284908893 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs") pod "network-metrics-daemon-6ghsl" (UID: "b9c4139c-2dde-4cd3-ba84-c9bee977f4ea") : secret "metrics-daemon-secret" not found
Sep 10 14:13:40.945077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:40.945042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:40.945077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:40.945084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:40.945510 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:40.945165    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:40.945510 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:40.945190    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:40.945510 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:40.945207    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:48.945195079 +0000 UTC m=+140.906704575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:40.945510 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:40.945238    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:13:48.94522611 +0000 UTC m=+140.906735634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:44.268525 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:44.268490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:13:44.268918 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:44.268635    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:44.268918 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:44.268712    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls podName:0f425bdc-f513-4b90-8fb3-3ebb7595f6a5 nodeName:}" failed. No retries permitted until 2026-09-10 14:14:00.268684231 +0000 UTC m=+152.230193732 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xq78" (UID: "0f425bdc-f513-4b90-8fb3-3ebb7595f6a5") : secret "cluster-monitoring-operator-tls" not found
Sep 10 14:13:49.004082 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:49.004053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:49.004082 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:49.004087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:13:49.004499 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:49.004197    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:49.004499 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:13:49.004261    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls podName:cd51d5fd-2863-44ce-82fc-f1da8a53857b nodeName:}" failed. No retries permitted until 2026-09-10 14:14:05.004243421 +0000 UTC m=+156.965752937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xzzw5" (UID: "cd51d5fd-2863-44ce-82fc-f1da8a53857b") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:49.004586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:13:49.004573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd51d5fd-2863-44ce-82fc-f1da8a53857b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:14:00.286810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:00.286757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:14:00.289343 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:00.289313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f425bdc-f513-4b90-8fb3-3ebb7595f6a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xq78\" (UID: \"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:14:00.584206 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:00.584135    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"
Sep 10 14:14:00.870777 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:00.870685    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78"]
Sep 10 14:14:00.873766 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:00.873733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f425bdc_f513_4b90_8fb3_3ebb7595f6a5.slice/crio-5a90d73e156d3b91b52bf3149f31223764096cd5cb83a5ab94ca80bb6557d0a3 WatchSource:0}: Error finding container 5a90d73e156d3b91b52bf3149f31223764096cd5cb83a5ab94ca80bb6557d0a3: Status 404 returned error can't find the container with id 5a90d73e156d3b91b52bf3149f31223764096cd5cb83a5ab94ca80bb6557d0a3
Sep 10 14:14:00.990727 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:00.990677    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78" event={"ID":"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5","Type":"ContainerStarted","Data":"5a90d73e156d3b91b52bf3149f31223764096cd5cb83a5ab94ca80bb6557d0a3"}
Sep 10 14:14:01.957987 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.957951    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"]
Sep 10 14:14:01.959955 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.959934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:01.963856 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.963828    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 14:14:01.963856 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.963833    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 14:14:01.964058 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.963947    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8c97b\""
Sep 10 14:14:01.964922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.964898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 14:14:01.965176 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.965114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 14:14:01.985606 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.985553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"]
Sep 10 14:14:01.999130 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.999106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5435054d-23dd-459b-9362-db71547432b3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:01.999262 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:01.999147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vzsf\" (UniqueName: \"kubernetes.io/projected/5435054d-23dd-459b-9362-db71547432b3-kube-api-access-7vzsf\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.063402 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.063372    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-z4gmp"]
Sep 10 14:14:02.065492 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.065471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.072454 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.072428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 14:14:02.072904 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.072727    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 14:14:02.073299 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.073165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pmtcp\""
Sep 10 14:14:02.098141 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.098118    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-z4gmp"]
Sep 10 14:14:02.100279 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.100258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5435054d-23dd-459b-9362-db71547432b3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.100381 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.100294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52422580-bfd8-4fec-b599-ab0a12145f18-nginx-conf\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.100381 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.100321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52422580-bfd8-4fec-b599-ab0a12145f18-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.100381 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.100362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vzsf\" (UniqueName: \"kubernetes.io/projected/5435054d-23dd-459b-9362-db71547432b3-kube-api-access-7vzsf\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.103318 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.103296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5435054d-23dd-459b-9362-db71547432b3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.148168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.148142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vzsf\" (UniqueName: \"kubernetes.io/projected/5435054d-23dd-459b-9362-db71547432b3-kube-api-access-7vzsf\") pod \"managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt\" (UID: \"5435054d-23dd-459b-9362-db71547432b3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.178884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.178856    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-2dlw6"]
Sep 10 14:14:02.181023 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.181002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:02.185049 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.185031    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 14:14:02.185565 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.185544    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 14:14:02.185726 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.185548    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kq6k5\""
Sep 10 14:14:02.201746 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.201727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52422580-bfd8-4fec-b599-ab0a12145f18-nginx-conf\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.201851 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.201752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52422580-bfd8-4fec-b599-ab0a12145f18-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.201851 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.201802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2gx\" (UniqueName: \"kubernetes.io/projected/ed9bbea1-85c9-4de0-b61b-50813c8d03e6-kube-api-access-6g2gx\") pod \"downloads-85d98c998f-2dlw6\" (UID: \"ed9bbea1-85c9-4de0-b61b-50813c8d03e6\") " pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:02.202421 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.202408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52422580-bfd8-4fec-b599-ab0a12145f18-nginx-conf\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.204338 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.204316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52422580-bfd8-4fec-b599-ab0a12145f18-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-z4gmp\" (UID: \"52422580-bfd8-4fec-b599-ab0a12145f18\") " pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.224940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.224896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-2dlw6"]
Sep 10 14:14:02.277319 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.277288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"
Sep 10 14:14:02.302850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.302823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2gx\" (UniqueName: \"kubernetes.io/projected/ed9bbea1-85c9-4de0-b61b-50813c8d03e6-kube-api-access-6g2gx\") pod \"downloads-85d98c998f-2dlw6\" (UID: \"ed9bbea1-85c9-4de0-b61b-50813c8d03e6\") " pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:02.347932 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.347895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2gx\" (UniqueName: \"kubernetes.io/projected/ed9bbea1-85c9-4de0-b61b-50813c8d03e6-kube-api-access-6g2gx\") pod \"downloads-85d98c998f-2dlw6\" (UID: \"ed9bbea1-85c9-4de0-b61b-50813c8d03e6\") " pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:02.375575 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.375544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp"
Sep 10 14:14:02.491098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.491023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:02.965531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.965474    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-z4gmp"]
Sep 10 14:14:02.968122 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:02.968089    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52422580_bfd8_4fec_b599_ab0a12145f18.slice/crio-3a432abd870e6b6d4005a2112853042892789d60d988dfc79f9278a718888b82 WatchSource:0}: Error finding container 3a432abd870e6b6d4005a2112853042892789d60d988dfc79f9278a718888b82: Status 404 returned error can't find the container with id 3a432abd870e6b6d4005a2112853042892789d60d988dfc79f9278a718888b82
Sep 10 14:14:02.968344 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.968323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt"]
Sep 10 14:14:02.973590 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:02.973564    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5435054d_23dd_459b_9362_db71547432b3.slice/crio-89ddca717b98c6eb09cd982b31a287ac9e0c719cbfba669587f77e774fafebea WatchSource:0}: Error finding container 89ddca717b98c6eb09cd982b31a287ac9e0c719cbfba669587f77e774fafebea: Status 404 returned error can't find the container with id 89ddca717b98c6eb09cd982b31a287ac9e0c719cbfba669587f77e774fafebea
Sep 10 14:14:02.974112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:02.974088    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-2dlw6"]
Sep 10 14:14:02.978815 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:02.978766    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9bbea1_85c9_4de0_b61b_50813c8d03e6.slice/crio-d1b0e7721d29ef506e70f767895e9bff3142d54db734cc3288979d0454237b22 WatchSource:0}: Error finding container d1b0e7721d29ef506e70f767895e9bff3142d54db734cc3288979d0454237b22: Status 404 returned error can't find the container with id d1b0e7721d29ef506e70f767895e9bff3142d54db734cc3288979d0454237b22
Sep 10 14:14:03.000671 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.000607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-2dlw6" event={"ID":"ed9bbea1-85c9-4de0-b61b-50813c8d03e6","Type":"ContainerStarted","Data":"d1b0e7721d29ef506e70f767895e9bff3142d54db734cc3288979d0454237b22"}
Sep 10 14:14:03.001815 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.001785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt" event={"ID":"5435054d-23dd-459b-9362-db71547432b3","Type":"ContainerStarted","Data":"89ddca717b98c6eb09cd982b31a287ac9e0c719cbfba669587f77e774fafebea"}
Sep 10 14:14:03.003237 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.003198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78" event={"ID":"0f425bdc-f513-4b90-8fb3-3ebb7595f6a5","Type":"ContainerStarted","Data":"c07bda9062087e60ba71fb110184730454133e9db2c7b8d45a8a2f7ffc0f87f4"}
Sep 10 14:14:03.004458 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.004434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp" event={"ID":"52422580-bfd8-4fec-b599-ab0a12145f18","Type":"ContainerStarted","Data":"3a432abd870e6b6d4005a2112853042892789d60d988dfc79f9278a718888b82"}
Sep 10 14:14:03.032161 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.032111    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xq78" podStartSLOduration=33.191682714 podStartE2EDuration="35.032098112s" totalImagesPullingTime="1.840415398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:03.030767003 +0000 UTC m=+154.992276531" watchObservedRunningTime="2026-09-10 14:14:03.032098112 +0000 UTC m=+154.993607630"
Sep 10 14:14:03.269053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.269028    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"]
Sep 10 14:14:03.271720 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.271688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:03.274453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.274432    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gl74f\""
Sep 10 14:14:03.274884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.274869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 14:14:03.281730 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.281687    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"]
Sep 10 14:14:03.313312 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.313289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gbxjz\" (UID: \"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:03.315234 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.315215    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:03.317123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.317108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.325880 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.325862    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 14:14:03.326106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.326088    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x4r25\""
Sep 10 14:14:03.326167 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.326152    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 14:14:03.327073 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.327057    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 14:14:03.329389 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.329374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 14:14:03.329559 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.329543    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 14:14:03.345607 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.345585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:03.413975 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.413944    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.413999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.414027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.414049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8h7f\" (UniqueName: \"kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.414112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gbxjz\" (UID: \"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:03.414250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.414136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.414159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.414250 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:03.414229    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 14:14:03.414439 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:03.414283    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates podName:e17d8a37-068e-4ee1-ad53-05b4fe22f1cd nodeName:}" failed. No retries permitted until 2026-09-10 14:14:03.914269202 +0000 UTC m=+155.875778703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-gbxjz" (UID: "e17d8a37-068e-4ee1-ad53-05b4fe22f1cd") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 14:14:03.514885 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.514857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.514885 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.514891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.514911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8h7f\" (UniqueName: \"kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.514955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.514979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.515039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515630 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.515609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515630 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.515622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.515784 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.515681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.517373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.517355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.517460 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.517392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.523408 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.523350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8h7f\" (UniqueName: \"kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f\") pod \"console-5db6d8d944-z2bk4\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") " pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.625084 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.625050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:03.909983 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.909908    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:03.913300 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:03.913263    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611bf8c9_e196_4c34_95da_066a315435eb.slice/crio-b14400be79c2dd09126bcb0d89ba743ecb570d107e4a2777153c0424fd3318a9 WatchSource:0}: Error finding container b14400be79c2dd09126bcb0d89ba743ecb570d107e4a2777153c0424fd3318a9: Status 404 returned error can't find the container with id b14400be79c2dd09126bcb0d89ba743ecb570d107e4a2777153c0424fd3318a9
Sep 10 14:14:03.918902 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.918874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gbxjz\" (UID: \"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:03.921884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:03.921857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e17d8a37-068e-4ee1-ad53-05b4fe22f1cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gbxjz\" (UID: \"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:04.009277 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:04.009244    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db6d8d944-z2bk4" event={"ID":"611bf8c9-e196-4c34-95da-066a315435eb","Type":"ContainerStarted","Data":"b14400be79c2dd09126bcb0d89ba743ecb570d107e4a2777153c0424fd3318a9"}
Sep 10 14:14:04.182256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:04.182180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:04.587653 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:04.587600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"]
Sep 10 14:14:04.590849 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:04.590822    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17d8a37_068e_4ee1_ad53_05b4fe22f1cd.slice/crio-980e099b9147967c25d945fb5d272fcd8f1ce762e854b18324a172297fbc34b1 WatchSource:0}: Error finding container 980e099b9147967c25d945fb5d272fcd8f1ce762e854b18324a172297fbc34b1: Status 404 returned error can't find the container with id 980e099b9147967c25d945fb5d272fcd8f1ce762e854b18324a172297fbc34b1
Sep 10 14:14:04.884816 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:04.884714    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c"
Sep 10 14:14:04.889890 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:04.889854    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2hd5g" podUID="68e8319c-15ef-4012-b590-a364beff74e7"
Sep 10 14:14:05.015207 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.015152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz" event={"ID":"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd","Type":"ContainerStarted","Data":"980e099b9147967c25d945fb5d272fcd8f1ce762e854b18324a172297fbc34b1"}
Sep 10 14:14:05.017054 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.017032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:05.017180 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.017115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp" event={"ID":"52422580-bfd8-4fec-b599-ab0a12145f18","Type":"ContainerStarted","Data":"10331144d92df952ef7dfb34f520b584eb74f8bce8b1ea6964326761a89faeda"}
Sep 10 14:14:05.017486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.017464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:05.029017 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.028990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:14:05.031355 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.031336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd51d5fd-2863-44ce-82fc-f1da8a53857b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xzzw5\" (UID: \"cd51d5fd-2863-44ce-82fc-f1da8a53857b\") " pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:14:05.035788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.035746    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-z4gmp" podStartSLOduration=1.663395719 podStartE2EDuration="3.035730243s" totalImagesPullingTime="1.372334524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:05.035474551 +0000 UTC m=+156.996984071" watchObservedRunningTime="2026-09-10 14:14:05.035730243 +0000 UTC m=+156.997239762"
Sep 10 14:14:05.255379 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.255306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xzzw5"
Sep 10 14:14:05.544897 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:05.544844    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6ghsl" podUID="b9c4139c-2dde-4cd3-ba84-c9bee977f4ea"
Sep 10 14:14:05.561559 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:05.561474    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xzzw5"]
Sep 10 14:14:05.726169 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:05.726133    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd51d5fd_2863_44ce_82fc_f1da8a53857b.slice/crio-9feb5fdbd9ded1929369540d10849a4d3e900a6c22b9fdf0295117849297b96d WatchSource:0}: Error finding container 9feb5fdbd9ded1929369540d10849a4d3e900a6c22b9fdf0295117849297b96d: Status 404 returned error can't find the container with id 9feb5fdbd9ded1929369540d10849a4d3e900a6c22b9fdf0295117849297b96d
Sep 10 14:14:06.023080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.023045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz" event={"ID":"e17d8a37-068e-4ee1-ad53-05b4fe22f1cd","Type":"ContainerStarted","Data":"560c4220ad8438e112c5d1cce677066c2f1b78ae718a93c75647d9db03eaa052"}
Sep 10 14:14:06.023553 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.023142    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:06.024857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.024826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xzzw5" event={"ID":"cd51d5fd-2863-44ce-82fc-f1da8a53857b","Type":"ContainerStarted","Data":"539689fdba0444ce67f4b3668e5313b664bf6ec210002f90c58c3a0aaf375130"}
Sep 10 14:14:06.025024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.024886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xzzw5" event={"ID":"cd51d5fd-2863-44ce-82fc-f1da8a53857b","Type":"ContainerStarted","Data":"9feb5fdbd9ded1929369540d10849a4d3e900a6c22b9fdf0295117849297b96d"}
Sep 10 14:14:06.029157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.029116    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz"
Sep 10 14:14:06.045196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:06.045105    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gbxjz" podStartSLOduration=1.859820312 podStartE2EDuration="3.045088026s" totalImagesPullingTime="1.185267714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:06.043265588 +0000 UTC m=+158.004775144" watchObservedRunningTime="2026-09-10 14:14:06.045088026 +0000 UTC m=+158.006597544"
Sep 10 14:14:08.035268 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:08.035226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db6d8d944-z2bk4" event={"ID":"611bf8c9-e196-4c34-95da-066a315435eb","Type":"ContainerStarted","Data":"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"}
Sep 10 14:14:08.037461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:08.037431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xzzw5" event={"ID":"cd51d5fd-2863-44ce-82fc-f1da8a53857b","Type":"ContainerStarted","Data":"0ffdd8690d5d27ac306bf812a57e53c6cf15ddc23dc18176e3710f60f8a8e710"}
Sep 10 14:14:08.078023 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:08.077978    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db6d8d944-z2bk4" podStartSLOduration=1.123998472 podStartE2EDuration="5.077962301s" totalImagesPullingTime="3.953963829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:08.076961453 +0000 UTC m=+160.038470973" watchObservedRunningTime="2026-09-10 14:14:08.077962301 +0000 UTC m=+160.039471812"
Sep 10 14:14:09.658853 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.658827    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:09.661928 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.661909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.670904 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.670876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 14:14:09.673341 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.673314    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:09.771197 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:09.771373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g9lq\" (UniqueName: \"kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:09.771596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.771596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.771577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.773783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.773761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68e8319c-15ef-4012-b590-a364beff74e7-metrics-tls\") pod \"dns-default-2hd5g\" (UID: \"68e8319c-15ef-4012-b590-a364beff74e7\") " pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:09.774168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.774138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"image-registry-77bd49b46f-xktx4\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") " pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:09.820685 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.820625    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wzphp\""
Sep 10 14:14:09.820834 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.820736    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwswr\""
Sep 10 14:14:09.828019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.827994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:09.828123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.828113    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:09.872771 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.872941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.872941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.872941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.872941 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.873153 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.873153 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.872974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g9lq\" (UniqueName: \"kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.873930 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.873903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.874039 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.873939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.874811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.874752    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.874811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.874769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.876353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.876304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.877098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.877076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.883719 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.883659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g9lq\" (UniqueName: \"kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq\") pod \"console-7c67dcd9fd-stht8\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") " pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:09.971258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:09.971212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:10.049854 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:10.049810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xzzw5" event={"ID":"cd51d5fd-2863-44ce-82fc-f1da8a53857b","Type":"ContainerStarted","Data":"e76608638933e71785880deb4aefaf8d66760c355f7455d025970bad1eac9199"}
Sep 10 14:14:10.067303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:10.067258    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xzzw5" podStartSLOduration=33.397097089 podStartE2EDuration="37.067242141s" totalImagesPullingTime="3.670145052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:13:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:10.066649611 +0000 UTC m=+162.028159130" watchObservedRunningTime="2026-09-10 14:14:10.067242141 +0000 UTC m=+162.028751659"
Sep 10 14:14:10.117274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:10.117216    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hd5g"]
Sep 10 14:14:10.120506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:10.120475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:14:10.120861 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:10.120834    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e8319c_15ef_4012_b590_a364beff74e7.slice/crio-0ca6073c2e5f8e883f3361c29a65e0d8327eb952789484c2df7d3ca505f1a0ed WatchSource:0}: Error finding container 0ca6073c2e5f8e883f3361c29a65e0d8327eb952789484c2df7d3ca505f1a0ed: Status 404 returned error can't find the container with id 0ca6073c2e5f8e883f3361c29a65e0d8327eb952789484c2df7d3ca505f1a0ed
Sep 10 14:14:10.123567 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:10.123478    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b18c3c_0e1b_486e_a94b_46355674210c.slice/crio-8fbdbc242f19755192166d77451514d30186aece71bce283d2e73554c7487194 WatchSource:0}: Error finding container 8fbdbc242f19755192166d77451514d30186aece71bce283d2e73554c7487194: Status 404 returned error can't find the container with id 8fbdbc242f19755192166d77451514d30186aece71bce283d2e73554c7487194
Sep 10 14:14:10.246537 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:10.246509    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:10.250056 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:10.250024    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4151dd37_25fe_47ca_a4ae_daf935b7b429.slice/crio-17569b689cf03d0890747e58217cb77d39c30f1347166400d4b72f93d072a3e8 WatchSource:0}: Error finding container 17569b689cf03d0890747e58217cb77d39c30f1347166400d4b72f93d072a3e8: Status 404 returned error can't find the container with id 17569b689cf03d0890747e58217cb77d39c30f1347166400d4b72f93d072a3e8
Sep 10 14:14:11.058936 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.058896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c67dcd9fd-stht8" event={"ID":"4151dd37-25fe-47ca-a4ae-daf935b7b429","Type":"ContainerStarted","Data":"0f3a25ecc619ceb94ff8f3e8ddf5a5c7c954725e64df28a2ede9799f6395f327"}
Sep 10 14:14:11.059378 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.058963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c67dcd9fd-stht8" event={"ID":"4151dd37-25fe-47ca-a4ae-daf935b7b429","Type":"ContainerStarted","Data":"17569b689cf03d0890747e58217cb77d39c30f1347166400d4b72f93d072a3e8"}
Sep 10 14:14:11.061404 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.061373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hd5g" event={"ID":"68e8319c-15ef-4012-b590-a364beff74e7","Type":"ContainerStarted","Data":"0ca6073c2e5f8e883f3361c29a65e0d8327eb952789484c2df7d3ca505f1a0ed"}
Sep 10 14:14:11.065167 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.065082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" event={"ID":"a1b18c3c-0e1b-486e-a94b-46355674210c","Type":"ContainerStarted","Data":"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"}
Sep 10 14:14:11.065167 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.065133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" event={"ID":"a1b18c3c-0e1b-486e-a94b-46355674210c","Type":"ContainerStarted","Data":"8fbdbc242f19755192166d77451514d30186aece71bce283d2e73554c7487194"}
Sep 10 14:14:11.065346 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.065302    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:11.110361 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.110308    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c67dcd9fd-stht8" podStartSLOduration=2.110289234 podStartE2EDuration="2.110289234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:11.083858589 +0000 UTC m=+163.045368107" watchObservedRunningTime="2026-09-10 14:14:11.110289234 +0000 UTC m=+163.071798755"
Sep 10 14:14:11.110788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.110726    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" podStartSLOduration=162.110714242 podStartE2EDuration="2m42.110714242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:11.110017808 +0000 UTC m=+163.071527334" watchObservedRunningTime="2026-09-10 14:14:11.110714242 +0000 UTC m=+163.072223766"
Sep 10 14:14:11.731601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.731568    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"]
Sep 10 14:14:11.738433 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.738410    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.742773 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.742585    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:14:11.742773 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.742633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xlv76\""
Sep 10 14:14:11.744038 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.743863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 14:14:11.744038 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.743926    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 14:14:11.755899 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.755836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"]
Sep 10 14:14:11.789300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.789273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.789422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.789318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tctln\" (UniqueName: \"kubernetes.io/projected/2886062c-df86-4d79-a826-2d4172bc0c45-kube-api-access-tctln\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.789422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.789356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.789422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.789390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2886062c-df86-4d79-a826-2d4172bc0c45-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.885296 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.885266    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-zb668"]
Sep 10 14:14:11.890446 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.890417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.890585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.890467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tctln\" (UniqueName: \"kubernetes.io/projected/2886062c-df86-4d79-a826-2d4172bc0c45-kube-api-access-tctln\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.890585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.890505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.890585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.890542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2886062c-df86-4d79-a826-2d4172bc0c45-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.890786 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:11.890595    2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 14:14:11.891110 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:11.891090    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls podName:2886062c-df86-4d79-a826-2d4172bc0c45 nodeName:}" failed. No retries permitted until 2026-09-10 14:14:12.390716198 +0000 UTC m=+164.352225709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-26lv9" (UID: "2886062c-df86-4d79-a826-2d4172bc0c45") : secret "openshift-state-metrics-tls" not found
Sep 10 14:14:11.892466 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.891914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2886062c-df86-4d79-a826-2d4172bc0c45-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.892650 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.892626    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8sv55"]
Sep 10 14:14:11.894266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.894214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.894584 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.894535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.898731 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.898460    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:14:11.898731 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.898569    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 14:14:11.899193 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.899038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t6qkp\""
Sep 10 14:14:11.899761 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.899684    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.902366 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.902342    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 14:14:11.903025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.903008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 14:14:11.903254 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.903226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 14:14:11.903585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.903564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z2qt2\""
Sep 10 14:14:11.909568 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.909550    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 14:14:11.913646 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.913627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tctln\" (UniqueName: \"kubernetes.io/projected/2886062c-df86-4d79-a826-2d4172bc0c45-kube-api-access-tctln\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:11.924940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.924877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-zb668"]
Sep 10 14:14:11.991859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-sys\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.991859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.991859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-root\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-metrics-client-ca\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.991943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-wtmp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crx4l\" (UniqueName: \"kubernetes.io/projected/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-api-access-crx4l\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.992112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-tls\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-textfile\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8057d5a6-5443-4139-962d-e10a82f3b8f2-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hsgp\" (UniqueName: \"kubernetes.io/projected/3d5c392b-926a-481f-b7ae-f069a4d6b908-kube-api-access-9hsgp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:11.992391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:11.992293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093507 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-metrics-client-ca\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-wtmp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crx4l\" (UniqueName: \"kubernetes.io/projected/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-api-access-crx4l\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-tls\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-textfile\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8057d5a6-5443-4139-962d-e10a82f3b8f2-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-wtmp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hsgp\" (UniqueName: \"kubernetes.io/projected/3d5c392b-926a-481f-b7ae-f069a4d6b908-kube-api-access-9hsgp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-sys\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-root\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.093965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.093960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-metrics-client-ca\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094597    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094631    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-textfile\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-root\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.094751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.094722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d5c392b-926a-481f-b7ae-f069a4d6b908-sys\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.095348 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.095080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.095348 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.095315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8057d5a6-5443-4139-962d-e10a82f3b8f2-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.096602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.096580    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-tls\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.096939 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.096892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.097423 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.097393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d5c392b-926a-481f-b7ae-f069a4d6b908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.097840 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.097796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.104805 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.104781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crx4l\" (UniqueName: \"kubernetes.io/projected/8057d5a6-5443-4139-962d-e10a82f3b8f2-kube-api-access-crx4l\") pod \"kube-state-metrics-74cb57df7-zb668\" (UID: \"8057d5a6-5443-4139-962d-e10a82f3b8f2\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.106869 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.106830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hsgp\" (UniqueName: \"kubernetes.io/projected/3d5c392b-926a-481f-b7ae-f069a4d6b908-kube-api-access-9hsgp\") pod \"node-exporter-8sv55\" (UID: \"3d5c392b-926a-481f-b7ae-f069a4d6b908\") " pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.215229 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.215116    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668"
Sep 10 14:14:12.232857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.232827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sv55"
Sep 10 14:14:12.397229 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.397190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:12.399933 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.399904    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2886062c-df86-4d79-a826-2d4172bc0c45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-26lv9\" (UID: \"2886062c-df86-4d79-a826-2d4172bc0c45\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:12.650763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:12.650655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"
Sep 10 14:14:13.626047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.626020    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:13.626310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.626077    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:13.627233 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.627205    2581 patch_prober.go:28] interesting pod/console-5db6d8d944-z2bk4 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body=
Sep 10 14:14:13.627289 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.627255    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5db6d8d944-z2bk4" podUID="611bf8c9-e196-4c34-95da-066a315435eb" containerName="console" probeResult="failure" output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused"
Sep 10 14:14:13.771998 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.771919    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-zb668"]
Sep 10 14:14:13.777566 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:13.777334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8057d5a6_5443_4139_962d_e10a82f3b8f2.slice/crio-53ffdd6c1c151af5ef9e70def99b457fbbdaaba86fb12c39c469b7042bfd0096 WatchSource:0}: Error finding container 53ffdd6c1c151af5ef9e70def99b457fbbdaaba86fb12c39c469b7042bfd0096: Status 404 returned error can't find the container with id 53ffdd6c1c151af5ef9e70def99b457fbbdaaba86fb12c39c469b7042bfd0096
Sep 10 14:14:13.777566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:13.777465    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9"]
Sep 10 14:14:13.780732 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:13.780544    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2886062c_df86_4d79_a826_2d4172bc0c45.slice/crio-c398e4de6b9b5bb98e4215e288e73e6fbb38e647de4e3076bf9f74e02521eb9b WatchSource:0}: Error finding container c398e4de6b9b5bb98e4215e288e73e6fbb38e647de4e3076bf9f74e02521eb9b: Status 404 returned error can't find the container with id c398e4de6b9b5bb98e4215e288e73e6fbb38e647de4e3076bf9f74e02521eb9b
Sep 10 14:14:14.080949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.080911    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sv55" event={"ID":"3d5c392b-926a-481f-b7ae-f069a4d6b908","Type":"ContainerStarted","Data":"f54b59fb87269d1cf7180bd97d0c2f2e72836e14e4ee27ed89b76ed9ca413d56"}
Sep 10 14:14:14.082739 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.082695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9" event={"ID":"2886062c-df86-4d79-a826-2d4172bc0c45","Type":"ContainerStarted","Data":"0893197cf67ac84b9482defd9193c1a4babe10da8e8aea59fe0ec1b55abb320c"}
Sep 10 14:14:14.082855 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.082767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9" event={"ID":"2886062c-df86-4d79-a826-2d4172bc0c45","Type":"ContainerStarted","Data":"c398e4de6b9b5bb98e4215e288e73e6fbb38e647de4e3076bf9f74e02521eb9b"}
Sep 10 14:14:14.084057 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.084022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668" event={"ID":"8057d5a6-5443-4139-962d-e10a82f3b8f2","Type":"ContainerStarted","Data":"53ffdd6c1c151af5ef9e70def99b457fbbdaaba86fb12c39c469b7042bfd0096"}
Sep 10 14:14:14.085647 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.085619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt" event={"ID":"5435054d-23dd-459b-9362-db71547432b3","Type":"ContainerStarted","Data":"457c679a54cb6a926197275432d748d6970bcf4847931ce6704d88878ff84f53"}
Sep 10 14:14:14.087599 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.087573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hd5g" event={"ID":"68e8319c-15ef-4012-b590-a364beff74e7","Type":"ContainerStarted","Data":"8780f336dc01f5c1f777b0ca7284502d317d4d517c40b9e4b1dc22586c8718fb"}
Sep 10 14:14:14.087685 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.087615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hd5g" event={"ID":"68e8319c-15ef-4012-b590-a364beff74e7","Type":"ContainerStarted","Data":"8db2fbefed802d5ca33c398eb7bfe802c55f24d73d91830e17baddabaf7e567f"}
Sep 10 14:14:14.087740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.087724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:14.109294 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.109254    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d9c6c6c88-95wmt" podStartSLOduration=2.588942301 podStartE2EDuration="13.109237205s" totalImagesPullingTime="10.520294904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:14.109107936 +0000 UTC m=+166.070617455" watchObservedRunningTime="2026-09-10 14:14:14.109237205 +0000 UTC m=+166.070746724"
Sep 10 14:14:14.140409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:14.140361    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2hd5g" podStartSLOduration=129.777784052 podStartE2EDuration="2m13.140346178s" totalImagesPullingTime="3.362562126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:14.139657142 +0000 UTC m=+166.101166662" watchObservedRunningTime="2026-09-10 14:14:14.140346178 +0000 UTC m=+166.101855696"
Sep 10 14:14:16.467073 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.467046    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"]
Sep 10 14:14:16.471826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.471807    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.476069 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.476047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pmfbk\""
Sep 10 14:14:16.476162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.476065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 14:14:16.481922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.481842    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:16.483417 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.483394    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"]
Sep 10 14:14:16.535875 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.535844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72hzj\" (UniqueName: \"kubernetes.io/projected/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-kube-api-access-72hzj\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.536004 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.535971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.636587 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.636556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72hzj\" (UniqueName: \"kubernetes.io/projected/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-kube-api-access-72hzj\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.636750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.636643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.639181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.639154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.646799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.646760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72hzj\" (UniqueName: \"kubernetes.io/projected/d183bbac-4cdd-4e24-b3fd-bee5b2b15f47-kube-api-access-72hzj\") pod \"monitoring-plugin-85bbbbbf4-8mcmh\" (UID: \"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:16.782984 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:16.782951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:17.939913 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.939879    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:14:17.943995 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.943974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:17.946583 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.946527    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 14:14:17.946750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.946602    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 14:14:17.946837 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.946758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9khrgb0qg9kuo\""
Sep 10 14:14:17.946837 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.946784    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 14:14:17.947908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.947876    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 14:14:17.948028 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.947945    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 14:14:17.948028 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.947989    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fhzrs\""
Sep 10 14:14:17.948166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948072    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 14:14:17.948166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 14:14:17.948370 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948176    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 14:14:17.948618 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948597    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 14:14:17.948750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948726    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 14:14:17.948826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948753    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 14:14:17.948884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 14:14:17.948977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.948963    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 14:14:17.951988 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.951967    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 14:14:17.955085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.955062    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 14:14:17.961296 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:17.961275    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:14:18.050882 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.050848    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.050898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.050961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bpcx\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.051651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.051490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.152935 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.152976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bpcx\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153814 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153814 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.153982 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.153949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.157715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.154115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.157715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.156449    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.157715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.156876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.157715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.157289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.157715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.157640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158034 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.157787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.158463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158725 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.158685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.158735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.158882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.158958 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.158881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.159333 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.159309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.160086 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.160045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.160487 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.160463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.160664 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.160637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.160898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.160880    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.161619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.161599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.164825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.164802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bpcx\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx\") pod \"prometheus-k8s-0\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:18.261029 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:18.260956    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:19.971808 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:19.971769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:19.972272 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:19.971914    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:19.973566 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:19.973539    2581 patch_prober.go:28] interesting pod/console-7c67dcd9fd-stht8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body=
Sep 10 14:14:19.973692 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:19.973599    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7c67dcd9fd-stht8" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused"
Sep 10 14:14:20.526076 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:20.526029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:14:21.050612 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.050537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"]
Sep 10 14:14:21.055604 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:21.055570    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd183bbac_4cdd_4e24_b3fd_bee5b2b15f47.slice/crio-c3d1b7ec97efc1214cc6155b64da1e497916296b438d73cc132206d24e60d95d WatchSource:0}: Error finding container c3d1b7ec97efc1214cc6155b64da1e497916296b438d73cc132206d24e60d95d: Status 404 returned error can't find the container with id c3d1b7ec97efc1214cc6155b64da1e497916296b438d73cc132206d24e60d95d
Sep 10 14:14:21.069181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.069080    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:14:21.076163 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:14:21.076131    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8f7469_07f1_47d4_9500_4d7150dc63d7.slice/crio-67ac9d3ba2c89ad285d7d96cc9bfc290ffaf1d731ba3802c983d749838b39f34 WatchSource:0}: Error finding container 67ac9d3ba2c89ad285d7d96cc9bfc290ffaf1d731ba3802c983d749838b39f34: Status 404 returned error can't find the container with id 67ac9d3ba2c89ad285d7d96cc9bfc290ffaf1d731ba3802c983d749838b39f34
Sep 10 14:14:21.118735 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.117751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-2dlw6" event={"ID":"ed9bbea1-85c9-4de0-b61b-50813c8d03e6","Type":"ContainerStarted","Data":"f0173e48efd6c866302ad572b90044bfb0daf46beadf67ad62fb357a37e74e8c"}
Sep 10 14:14:21.118866 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.118770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:21.120812 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.120783    2581 patch_prober.go:28] interesting pod/downloads-85d98c998f-2dlw6 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused" start-of-body=
Sep 10 14:14:21.120927 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.120844    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-2dlw6" podUID="ed9bbea1-85c9-4de0-b61b-50813c8d03e6" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused"
Sep 10 14:14:21.123068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.123042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9" event={"ID":"2886062c-df86-4d79-a826-2d4172bc0c45","Type":"ContainerStarted","Data":"b80e6015cc7d67ad59b061c2cf7feb87d739ed8f8e5a234c496b359b9742c7a1"}
Sep 10 14:14:21.125138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.125113    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"67ac9d3ba2c89ad285d7d96cc9bfc290ffaf1d731ba3802c983d749838b39f34"}
Sep 10 14:14:21.128036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.128010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh" event={"ID":"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47","Type":"ContainerStarted","Data":"c3d1b7ec97efc1214cc6155b64da1e497916296b438d73cc132206d24e60d95d"}
Sep 10 14:14:21.141131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:21.141007    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-2dlw6" podStartSLOduration=1.248486201 podStartE2EDuration="19.140988244s" totalImagesPullingTime="17.892502043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:21.138217331 +0000 UTC m=+173.099726849" watchObservedRunningTime="2026-09-10 14:14:21.140988244 +0000 UTC m=+173.102497765"
Sep 10 14:14:22.145071 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:22.145004    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-2dlw6"
Sep 10 14:14:23.139412 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:23.139243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sv55" event={"ID":"3d5c392b-926a-481f-b7ae-f069a4d6b908","Type":"ContainerStarted","Data":"825f4faa0def1d0dd94c8356d9fcc7ec60a628a6041e738b3463aaa6786f2b37"}
Sep 10 14:14:23.142354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:23.142320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668" event={"ID":"8057d5a6-5443-4139-962d-e10a82f3b8f2","Type":"ContainerStarted","Data":"4db2b8be9978f347f3f4c5dc8f599a80414ab3766cfb80161327cf60f2769193"}
Sep 10 14:14:23.142505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:23.142383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668" event={"ID":"8057d5a6-5443-4139-962d-e10a82f3b8f2","Type":"ContainerStarted","Data":"bf21c7203a336f1dadd160a378b17f7838f2b8833eeabf57e1c539e6daa59bbb"}
Sep 10 14:14:24.093406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:24.093376    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2hd5g"
Sep 10 14:14:24.148876 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:24.148550    2581 generic.go:356] "Generic (PLEG): container finished" podID="3d5c392b-926a-481f-b7ae-f069a4d6b908" containerID="825f4faa0def1d0dd94c8356d9fcc7ec60a628a6041e738b3463aaa6786f2b37" exitCode=0
Sep 10 14:14:24.148876 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:24.148594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sv55" event={"ID":"3d5c392b-926a-481f-b7ae-f069a4d6b908","Type":"ContainerDied","Data":"825f4faa0def1d0dd94c8356d9fcc7ec60a628a6041e738b3463aaa6786f2b37"}
Sep 10 14:14:25.156363 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.156323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sv55" event={"ID":"3d5c392b-926a-481f-b7ae-f069a4d6b908","Type":"ContainerStarted","Data":"2017a38a2bfc1ab7bb40d5b96a083b63200a6003c4901eacea9d46272fb75671"}
Sep 10 14:14:25.156827 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.156392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sv55" event={"ID":"3d5c392b-926a-481f-b7ae-f069a4d6b908","Type":"ContainerStarted","Data":"8f5ee68fe45183567f3d2e5dbaf0f0a0c4cd85b86cf3c016073d6c26a78dcd46"}
Sep 10 14:14:25.158897 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.158866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9" event={"ID":"2886062c-df86-4d79-a826-2d4172bc0c45","Type":"ContainerStarted","Data":"1f5b11857c429bf3af96a6db46ef96327899bea91073f84295317f4935a5c502"}
Sep 10 14:14:25.160777 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.160751    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="dc9fb559e44f688bc6614f252707d78142615f661eb5dffab8c41e93427ef659" exitCode=0
Sep 10 14:14:25.160874 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.160839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"dc9fb559e44f688bc6614f252707d78142615f661eb5dffab8c41e93427ef659"}
Sep 10 14:14:25.162651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.162627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh" event={"ID":"d183bbac-4cdd-4e24-b3fd-bee5b2b15f47","Type":"ContainerStarted","Data":"9281402192f836b6c9a3d2efd45c13519d0cb9f6ed0584f9761e75f28d339115"}
Sep 10 14:14:25.162827 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.162806    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:25.165463 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.165434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668" event={"ID":"8057d5a6-5443-4139-962d-e10a82f3b8f2","Type":"ContainerStarted","Data":"4205f1290c1a1e6403a15b19029fd8ccc54b010c5a87a26c1f97c73d617d7f1a"}
Sep 10 14:14:25.168227 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.168208    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh"
Sep 10 14:14:25.194621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.194579    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8sv55" podStartSLOduration=4.982241313 podStartE2EDuration="14.194564736s" totalImagesPullingTime="9.070337179s" totalInitContainerRuntime="141.986244ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:25.178286747 +0000 UTC m=+177.139796266" watchObservedRunningTime="2026-09-10 14:14:25.194564736 +0000 UTC m=+177.156074257"
Sep 10 14:14:25.194792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.194767    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-8mcmh" podStartSLOduration=6.054808728 podStartE2EDuration="9.194758168s" totalImagesPullingTime="3.13994944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:25.193223719 +0000 UTC m=+177.154733251" watchObservedRunningTime="2026-09-10 14:14:25.194758168 +0000 UTC m=+177.156267687"
Sep 10 14:14:25.239725 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.239654    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-26lv9" podStartSLOduration=10.893786183 podStartE2EDuration="14.239629655s" totalImagesPullingTime="3.345843472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:25.238644098 +0000 UTC m=+177.200153617" watchObservedRunningTime="2026-09-10 14:14:25.239629655 +0000 UTC m=+177.201139174"
Sep 10 14:14:25.258384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:25.258341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-zb668" podStartSLOduration=5.469769866 podStartE2EDuration="14.258326221s" totalImagesPullingTime="8.788556355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:25.256986208 +0000 UTC m=+177.218495739" watchObservedRunningTime="2026-09-10 14:14:25.258326221 +0000 UTC m=+177.219835739"
Sep 10 14:14:27.066086 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:27.066054    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:29.833794 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:29.833763    2581 patch_prober.go:28] interesting pod/image-registry-77bd49b46f-xktx4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 14:14:29.834140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:29.833834    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:14:30.198102 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:30.198059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"69c5620a715785e32ffcfbded2b29f21ca74e0ab5edde42896f82ca6ee244a03"}
Sep 10 14:14:30.198350 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:30.198186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"54985988a1684749f0ef85695be0ad1e780bc80106a98e392fca5a5a8b25cb9f"}
Sep 10 14:14:32.074718 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:32.074677    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:33.214176 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.214112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"cf0fe759eef50560b0e663994568c5dc1062e636d6b6c71187d676e2b5cde17a"}
Sep 10 14:14:33.214176 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.214173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"5acf433c97a0b1734c5e9490da811db48f01a8a2fd4a19cf5cdbb5e385d5e329"}
Sep 10 14:14:33.214549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.214201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"cd11cceff58362ffc6ca58edf1481d85565ce5222ffaba7e12eedd60b5eca61e"}
Sep 10 14:14:33.214549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.214224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerStarted","Data":"112ff041531c4481780d9356c18d9c8fafb5cef3a373438d76fa20c8e1013ebb"}
Sep 10 14:14:33.243771 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.243693    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.924872982 podStartE2EDuration="16.243677397s" totalImagesPullingTime="10.304269675s" totalInitContainerRuntime="14.53474ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:33.242738728 +0000 UTC m=+185.204248247" watchObservedRunningTime="2026-09-10 14:14:33.243677397 +0000 UTC m=+185.205186966"
Sep 10 14:14:33.261359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.261337    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:14:33.939278 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:33.939248    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:14:41.504403 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.504337    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5db6d8d944-z2bk4" podUID="611bf8c9-e196-4c34-95da-066a315435eb" containerName="console" containerID="cri-o://e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca" gracePeriod=15
Sep 10 14:14:41.772506 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.772483    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:41.870528 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.870670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870535    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.870670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870563    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.870670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8h7f\" (UniqueName: \"kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.870670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870626    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.870916 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870763    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config\") pod \"611bf8c9-e196-4c34-95da-066a315435eb\" (UID: \"611bf8c9-e196-4c34-95da-066a315435eb\") "
Sep 10 14:14:41.871035 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870976    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:41.871035 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.870985    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:41.871035 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.871003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config" (OuterVolumeSpecName: "console-config") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:41.871249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.871125    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-oauth-serving-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:41.871249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.871141    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:41.871249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.871153    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/611bf8c9-e196-4c34-95da-066a315435eb-console-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:41.872801 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.872779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:41.872881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.872825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:41.872881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.872841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f" (OuterVolumeSpecName: "kube-api-access-j8h7f") pod "611bf8c9-e196-4c34-95da-066a315435eb" (UID: "611bf8c9-e196-4c34-95da-066a315435eb"). InnerVolumeSpecName "kube-api-access-j8h7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:14:41.972532 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.972506    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-serving-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:41.972532 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.972531    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8h7f\" (UniqueName: \"kubernetes.io/projected/611bf8c9-e196-4c34-95da-066a315435eb-kube-api-access-j8h7f\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:41.972681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:41.972541    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/611bf8c9-e196-4c34-95da-066a315435eb-console-oauth-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:42.247998 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.247916    2581 generic.go:356] "Generic (PLEG): container finished" podID="611bf8c9-e196-4c34-95da-066a315435eb" containerID="e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca" exitCode=0
Sep 10 14:14:42.247998 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.247964    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db6d8d944-z2bk4"
Sep 10 14:14:42.248163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.248005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db6d8d944-z2bk4" event={"ID":"611bf8c9-e196-4c34-95da-066a315435eb","Type":"ContainerDied","Data":"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"}
Sep 10 14:14:42.248163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.248056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db6d8d944-z2bk4" event={"ID":"611bf8c9-e196-4c34-95da-066a315435eb","Type":"ContainerDied","Data":"b14400be79c2dd09126bcb0d89ba743ecb570d107e4a2777153c0424fd3318a9"}
Sep 10 14:14:42.248163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.248078    2581 scope.go:117] "RemoveContainer" containerID="e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"
Sep 10 14:14:42.262826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.262807    2581 scope.go:117] "RemoveContainer" containerID="e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"
Sep 10 14:14:42.263094 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:42.263075    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca\": container with ID starting with e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca not found: ID does not exist" containerID="e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"
Sep 10 14:14:42.263139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.263103    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca"} err="failed to get container status \"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca\": rpc error: code = NotFound desc = could not find container \"e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca\": container with ID starting with e61abe283949feb9c67f4a4d56d7fdbf68780de4ced7bf3f115b198225575eca not found: ID does not exist"
Sep 10 14:14:42.271414 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.268113    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:42.272619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.272484    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db6d8d944-z2bk4"]
Sep 10 14:14:42.530457 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:42.530414    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611bf8c9-e196-4c34-95da-066a315435eb" path="/var/lib/kubelet/pods/611bf8c9-e196-4c34-95da-066a315435eb/volumes"
Sep 10 14:14:52.094626 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.094587    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c67dcd9fd-stht8" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerName="console" containerID="cri-o://0f3a25ecc619ceb94ff8f3e8ddf5a5c7c954725e64df28a2ede9799f6395f327" gracePeriod=15
Sep 10 14:14:52.289098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.289061    2581 generic.go:356] "Generic (PLEG): container finished" podID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerID="0f3a25ecc619ceb94ff8f3e8ddf5a5c7c954725e64df28a2ede9799f6395f327" exitCode=0
Sep 10 14:14:52.289229 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.289126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c67dcd9fd-stht8" event={"ID":"4151dd37-25fe-47ca-a4ae-daf935b7b429","Type":"ContainerDied","Data":"0f3a25ecc619ceb94ff8f3e8ddf5a5c7c954725e64df28a2ede9799f6395f327"}
Sep 10 14:14:52.333638 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.333612    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:52.350776 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.350891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g9lq\" (UniqueName: \"kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.350891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.350891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.350891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350875    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.351085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350918    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.351085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.350953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle\") pod \"4151dd37-25fe-47ca-a4ae-daf935b7b429\" (UID: \"4151dd37-25fe-47ca-a4ae-daf935b7b429\") "
Sep 10 14:14:52.351085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.351063    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:52.351621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.351336    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-oauth-serving-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.351621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.351408    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:52.351621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.351512    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca" (OuterVolumeSpecName: "service-ca") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:52.351850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.351634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config" (OuterVolumeSpecName: "console-config") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:52.353406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.353338    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq" (OuterVolumeSpecName: "kube-api-access-2g9lq") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "kube-api-access-2g9lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:14:52.354256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.354227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:52.354692 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.354670    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4151dd37-25fe-47ca-a4ae-daf935b7b429" (UID: "4151dd37-25fe-47ca-a4ae-daf935b7b429"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:52.452070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452040    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-serving-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.452070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452065    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-trusted-ca-bundle\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.452070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452076    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g9lq\" (UniqueName: \"kubernetes.io/projected/4151dd37-25fe-47ca-a4ae-daf935b7b429-kube-api-access-2g9lq\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.452269 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452084    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.452269 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452093    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4151dd37-25fe-47ca-a4ae-daf935b7b429-console-oauth-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:52.452269 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:52.452101    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4151dd37-25fe-47ca-a4ae-daf935b7b429-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:53.293765 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.293729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c67dcd9fd-stht8" event={"ID":"4151dd37-25fe-47ca-a4ae-daf935b7b429","Type":"ContainerDied","Data":"17569b689cf03d0890747e58217cb77d39c30f1347166400d4b72f93d072a3e8"}
Sep 10 14:14:53.293765 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.293754    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c67dcd9fd-stht8"
Sep 10 14:14:53.294245 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.293790    2581 scope.go:117] "RemoveContainer" containerID="0f3a25ecc619ceb94ff8f3e8ddf5a5c7c954725e64df28a2ede9799f6395f327"
Sep 10 14:14:53.295440 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.295415    2581 generic.go:356] "Generic (PLEG): container finished" podID="1c283e68-f138-4300-8e04-bb7fb2727d30" containerID="cfcce862de28b4927f0429c88ffe77eb08c275c02f1a969b0490b43b04801a8b" exitCode=0
Sep 10 14:14:53.295578 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.295487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c" event={"ID":"1c283e68-f138-4300-8e04-bb7fb2727d30","Type":"ContainerDied","Data":"cfcce862de28b4927f0429c88ffe77eb08c275c02f1a969b0490b43b04801a8b"}
Sep 10 14:14:53.295983 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.295888    2581 scope.go:117] "RemoveContainer" containerID="cfcce862de28b4927f0429c88ffe77eb08c275c02f1a969b0490b43b04801a8b"
Sep 10 14:14:53.332584 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.332558    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:53.338840 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:53.338820    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c67dcd9fd-stht8"]
Sep 10 14:14:54.301402 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:54.301360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-22b5c" event={"ID":"1c283e68-f138-4300-8e04-bb7fb2727d30","Type":"ContainerStarted","Data":"cec4af7b32376b108a96d5737aa3590f6a9c2e76406ebe4fc6fccaa8fe4babbb"}
Sep 10 14:14:54.530505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:54.530472    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" path="/var/lib/kubelet/pods/4151dd37-25fe-47ca-a4ae-daf935b7b429/volumes"
Sep 10 14:14:58.958496 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:58.958451    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerName="registry" containerID="cri-o://e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b" gracePeriod=30
Sep 10 14:14:59.237210 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.237158    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:59.309000 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.308970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd5w6\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309045    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309092    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309140 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309416 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309157    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309416 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309196    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates\") pod \"a1b18c3c-0e1b-486e-a94b-46355674210c\" (UID: \"a1b18c3c-0e1b-486e-a94b-46355674210c\") "
Sep 10 14:14:59.309659 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309631    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:59.309769 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.309676    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:14:59.311495 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.311469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:59.311590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.311540    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6" (OuterVolumeSpecName: "kube-api-access-vd5w6") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "kube-api-access-vd5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:14:59.311590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.311544    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:14:59.311943 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.311919    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:14:59.312019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.311949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:14:59.317305 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.317277    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a1b18c3c-0e1b-486e-a94b-46355674210c" (UID: "a1b18c3c-0e1b-486e-a94b-46355674210c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:14:59.324182 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.324158    2581 generic.go:356] "Generic (PLEG): container finished" podID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerID="e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b" exitCode=0
Sep 10 14:14:59.324259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.324195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" event={"ID":"a1b18c3c-0e1b-486e-a94b-46355674210c","Type":"ContainerDied","Data":"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"}
Sep 10 14:14:59.324259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.324232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4" event={"ID":"a1b18c3c-0e1b-486e-a94b-46355674210c","Type":"ContainerDied","Data":"8fbdbc242f19755192166d77451514d30186aece71bce283d2e73554c7487194"}
Sep 10 14:14:59.324259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.324252    2581 scope.go:117] "RemoveContainer" containerID="e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"
Sep 10 14:14:59.324364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.324198    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77bd49b46f-xktx4"
Sep 10 14:14:59.331609 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.331579    2581 scope.go:117] "RemoveContainer" containerID="e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"
Sep 10 14:14:59.332046 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:14:59.332018    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b\": container with ID starting with e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b not found: ID does not exist" containerID="e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"
Sep 10 14:14:59.332128 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.332057    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b"} err="failed to get container status \"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b\": rpc error: code = NotFound desc = could not find container \"e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b\": container with ID starting with e396524a277510ccf792f13a6abb5209efb2541df79da30c3374e17a52c6405b not found: ID does not exist"
Sep 10 14:14:59.344200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.344178    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:14:59.347243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.347225    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77bd49b46f-xktx4"]
Sep 10 14:14:59.409898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409871    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-image-registry-private-configuration\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.409898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409890    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-trusted-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.409898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409901    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b18c3c-0e1b-486e-a94b-46355674210c-installation-pull-secrets\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.410106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409910    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b18c3c-0e1b-486e-a94b-46355674210c-ca-trust-extracted\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.410106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409918    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-bound-sa-token\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.410106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409926    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.410106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409933    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b18c3c-0e1b-486e-a94b-46355674210c-registry-certificates\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:14:59.410106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:14:59.409940    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd5w6\" (UniqueName: \"kubernetes.io/projected/a1b18c3c-0e1b-486e-a94b-46355674210c-kube-api-access-vd5w6\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:00.529881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:00.529843    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" path="/var/lib/kubelet/pods/a1b18c3c-0e1b-486e-a94b-46355674210c/volumes"
Sep 10 14:15:18.261825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:18.261789    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:18.281024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:18.280992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:18.416666 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:18.416640    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:36.285681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.285644    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.285956    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="prometheus" containerID="cri-o://54985988a1684749f0ef85695be0ad1e780bc80106a98e392fca5a5a8b25cb9f" gracePeriod=600
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.285981    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="thanos-sidecar" containerID="cri-o://112ff041531c4481780d9356c18d9c8fafb5cef3a373438d76fa20c8e1013ebb" gracePeriod=600
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.286010    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="config-reloader" containerID="cri-o://69c5620a715785e32ffcfbded2b29f21ca74e0ab5edde42896f82ca6ee244a03" gracePeriod=600
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.285956    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy" containerID="cri-o://5acf433c97a0b1734c5e9490da811db48f01a8a2fd4a19cf5cdbb5e385d5e329" gracePeriod=600
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.286024    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-web" containerID="cri-o://cd11cceff58362ffc6ca58edf1481d85565ce5222ffaba7e12eedd60b5eca61e" gracePeriod=600
Sep 10 14:15:36.286226 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.286029    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://cf0fe759eef50560b0e663994568c5dc1062e636d6b6c71187d676e2b5cde17a" gracePeriod=600
Sep 10 14:15:36.470968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470934    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="cf0fe759eef50560b0e663994568c5dc1062e636d6b6c71187d676e2b5cde17a" exitCode=0
Sep 10 14:15:36.470968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470963    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="5acf433c97a0b1734c5e9490da811db48f01a8a2fd4a19cf5cdbb5e385d5e329" exitCode=0
Sep 10 14:15:36.470968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470969    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="cd11cceff58362ffc6ca58edf1481d85565ce5222ffaba7e12eedd60b5eca61e" exitCode=0
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470976    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="112ff041531c4481780d9356c18d9c8fafb5cef3a373438d76fa20c8e1013ebb" exitCode=0
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470986    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="69c5620a715785e32ffcfbded2b29f21ca74e0ab5edde42896f82ca6ee244a03" exitCode=0
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.470995    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerID="54985988a1684749f0ef85695be0ad1e780bc80106a98e392fca5a5a8b25cb9f" exitCode=0
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"cf0fe759eef50560b0e663994568c5dc1062e636d6b6c71187d676e2b5cde17a"}
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"5acf433c97a0b1734c5e9490da811db48f01a8a2fd4a19cf5cdbb5e385d5e329"}
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"cd11cceff58362ffc6ca58edf1481d85565ce5222ffaba7e12eedd60b5eca61e"}
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"112ff041531c4481780d9356c18d9c8fafb5cef3a373438d76fa20c8e1013ebb"}
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"69c5620a715785e32ffcfbded2b29f21ca74e0ab5edde42896f82ca6ee244a03"}
Sep 10 14:15:36.471232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.471208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"54985988a1684749f0ef85695be0ad1e780bc80106a98e392fca5a5a8b25cb9f"}
Sep 10 14:15:36.515442 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.515421    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:36.686232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686196    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686213    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686231    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686512    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686534    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686558    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bpcx\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.686657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686646    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686694    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.686866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\" (UID: \"9e8f7469-07f1-47d4-9500-4d7150dc63d7\") "
Sep 10 14:15:36.687335 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.687126    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.687335 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.687275    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.688141 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.688106    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.688461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.688431    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.689215 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.689186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.689572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.689546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.689672 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.689635    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:15:36.689786 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.689763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config" (OuterVolumeSpecName: "config") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.689883 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.689857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:15:36.690054 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.690121 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690055    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.690370 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690330    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.690575 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690544    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.690575 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690550    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out" (OuterVolumeSpecName: "config-out") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:15:36.690807 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690778    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.690945 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.690924    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:15:36.691194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.691171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.691631 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.691613    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx" (OuterVolumeSpecName: "kube-api-access-5bpcx") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "kube-api-access-5bpcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:15:36.692027 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.692006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.700440 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.700421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config" (OuterVolumeSpecName: "web-config") pod "9e8f7469-07f1-47d4-9500-4d7150dc63d7" (UID: "9e8f7469-07f1-47d4-9500-4d7150dc63d7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:15:36.787920 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787894    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-tls-assets\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.787920 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787919    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787933    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-web-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787946    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787958    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787970    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787982    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-grpc-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.787994    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bpcx\" (UniqueName: \"kubernetes.io/projected/9e8f7469-07f1-47d4-9500-4d7150dc63d7-kube-api-access-5bpcx\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788006    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-metrics-client-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788019    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788032    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-db\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788044    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788058    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-metrics-client-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788071    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e8f7469-07f1-47d4-9500-4d7150dc63d7-config-out\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788084    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788097    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788112    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788126    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e8f7469-07f1-47d4-9500-4d7150dc63d7-secret-kube-rbac-proxy\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:36.788523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:36.788139    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e8f7469-07f1-47d4-9500-4d7150dc63d7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:15:37.478093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.478057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e8f7469-07f1-47d4-9500-4d7150dc63d7","Type":"ContainerDied","Data":"67ac9d3ba2c89ad285d7d96cc9bfc290ffaf1d731ba3802c983d749838b39f34"}
Sep 10 14:15:37.478517 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.478102    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.478517 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.478147    2581 scope.go:117] "RemoveContainer" containerID="cf0fe759eef50560b0e663994568c5dc1062e636d6b6c71187d676e2b5cde17a"
Sep 10 14:15:37.486405 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.486387    2581 scope.go:117] "RemoveContainer" containerID="5acf433c97a0b1734c5e9490da811db48f01a8a2fd4a19cf5cdbb5e385d5e329"
Sep 10 14:15:37.493537 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.493522    2581 scope.go:117] "RemoveContainer" containerID="cd11cceff58362ffc6ca58edf1481d85565ce5222ffaba7e12eedd60b5eca61e"
Sep 10 14:15:37.499747 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.499731    2581 scope.go:117] "RemoveContainer" containerID="112ff041531c4481780d9356c18d9c8fafb5cef3a373438d76fa20c8e1013ebb"
Sep 10 14:15:37.503129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.503107    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:37.506375 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.506348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:37.506829 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.506813    2581 scope.go:117] "RemoveContainer" containerID="69c5620a715785e32ffcfbded2b29f21ca74e0ab5edde42896f82ca6ee244a03"
Sep 10 14:15:37.512933 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.512914    2581 scope.go:117] "RemoveContainer" containerID="54985988a1684749f0ef85695be0ad1e780bc80106a98e392fca5a5a8b25cb9f"
Sep 10 14:15:37.519324 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.519308    2581 scope.go:117] "RemoveContainer" containerID="dc9fb559e44f688bc6614f252707d78142615f661eb5dffab8c41e93427ef659"
Sep 10 14:15:37.536142 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536123    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:37.536356 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536345    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="thanos-sidecar"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536358    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="thanos-sidecar"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536374    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerName="console"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536379    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerName="console"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536386    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="init-config-reloader"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536391    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="init-config-reloader"
Sep 10 14:15:37.536401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536399    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-thanos"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536404    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-thanos"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536413    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-web"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536417    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-web"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536424    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="config-reloader"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536429    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="config-reloader"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536437    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="prometheus"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536442    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="prometheus"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536447    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536452    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536459    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611bf8c9-e196-4c34-95da-066a315435eb" containerName="console"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536464    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="611bf8c9-e196-4c34-95da-066a315435eb" containerName="console"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536470    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerName="registry"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536474    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerName="registry"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536516    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="thanos-sidecar"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536522    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536530    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-thanos"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536536    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="config-reloader"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536542    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="kube-rbac-proxy-web"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536548    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b18c3c-0e1b-486e-a94b-46355674210c" containerName="registry"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536553    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4151dd37-25fe-47ca-a4ae-daf935b7b429" containerName="console"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536561    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="611bf8c9-e196-4c34-95da-066a315435eb" containerName="console"
Sep 10 14:15:37.536569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.536566    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" containerName="prometheus"
Sep 10 14:15:37.540637 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.540622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.543322 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 14:15:37.543322 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543309    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 14:15:37.543464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543411    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 14:15:37.543464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 14:15:37.543571 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543551    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9khrgb0qg9kuo\""
Sep 10 14:15:37.543626 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543570    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 14:15:37.543687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543632    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 14:15:37.543687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543663    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 14:15:37.543813 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543749    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 14:15:37.543813 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543799    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 14:15:37.543927 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543829    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 14:15:37.543976 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.543968    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 14:15:37.544578 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.544514    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 14:15:37.544675 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.544617    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fhzrs\""
Sep 10 14:15:37.544833 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.544815    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 14:15:37.547302 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.547278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 14:15:37.548711 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.548675    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 14:15:37.556733 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.556681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:37.692490 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692490 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62xm\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-kube-api-access-w62xm\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692962 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692962 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.692962 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693054 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.692979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693150 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693150 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693210 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693210 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.693267 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.693215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794338 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w62xm\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-kube-api-access-w62xm\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794632 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794632 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794632 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.794810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.794969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795025 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.795017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795362 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.795048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.795600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.795576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.796385 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.796359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-config-out\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.797862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.798019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.797882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.798574 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.798503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.798939 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.798916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.799014 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.798940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.799961 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.799940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.800047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.800012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.800234 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.800219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.800379 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.800363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-web-config\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.800635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.800617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6dde411f-5379-41ab-a182-8502fdde692f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.801355 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.801335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6dde411f-5379-41ab-a182-8502fdde692f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.802429 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.802414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62xm\" (UniqueName: \"kubernetes.io/projected/6dde411f-5379-41ab-a182-8502fdde692f-kube-api-access-w62xm\") pod \"prometheus-k8s-0\" (UID: \"6dde411f-5379-41ab-a182-8502fdde692f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:37.850197 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:37.850143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:38.130826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:38.130804    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 14:15:38.132424 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:15:38.132402    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dde411f_5379_41ab_a182_8502fdde692f.slice/crio-1b1b0a79457904f349725a13556fe62dca213995e799b91d18961c7339198683 WatchSource:0}: Error finding container 1b1b0a79457904f349725a13556fe62dca213995e799b91d18961c7339198683: Status 404 returned error can't find the container with id 1b1b0a79457904f349725a13556fe62dca213995e799b91d18961c7339198683
Sep 10 14:15:38.483818 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:38.483736    2581 generic.go:356] "Generic (PLEG): container finished" podID="6dde411f-5379-41ab-a182-8502fdde692f" containerID="f7f62aa9359ba3067414801397942aafc2eabad3565a0d62e626f3d1b8e546fc" exitCode=0
Sep 10 14:15:38.483818 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:38.483777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerDied","Data":"f7f62aa9359ba3067414801397942aafc2eabad3565a0d62e626f3d1b8e546fc"}
Sep 10 14:15:38.484191 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:38.483825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"1b1b0a79457904f349725a13556fe62dca213995e799b91d18961c7339198683"}
Sep 10 14:15:38.530760 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:38.530733    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8f7469-07f1-47d4-9500-4d7150dc63d7" path="/var/lib/kubelet/pods/9e8f7469-07f1-47d4-9500-4d7150dc63d7/volumes"
Sep 10 14:15:39.491337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"4b365e9f398f866933acf5aed6d735201e9eecbf0c2636736035bcdc88f16ec5"}
Sep 10 14:15:39.491759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"6dffab4278e8759420588ef52787df39672e5f6dc115fd4058fd09945fcb4cb5"}
Sep 10 14:15:39.491759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"8c9e2481e54286b2dd72c87ccb209cb9f71b1828344cd708eedcf8a826e7149a"}
Sep 10 14:15:39.491759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"2d12ebd0ee7e5d7af045b72935d4ab6d1cc19aa1dfae8d0952c938c0d7c8db95"}
Sep 10 14:15:39.491759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"d6fdd1adf096ff4cca2f5fc89dab1e2903771d7f9eafe4ba4a3a15409bdac7fa"}
Sep 10 14:15:39.491759 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.491542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6dde411f-5379-41ab-a182-8502fdde692f","Type":"ContainerStarted","Data":"a14c63900544bbb924a6933cddd8e8f509c9c887843aea3cd0aaa938060a9c17"}
Sep 10 14:15:39.519862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:39.519811    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.50358066 podStartE2EDuration="2.519796273s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.215613ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:15:39.518083807 +0000 UTC m=+251.479593315" watchObservedRunningTime="2026-09-10 14:15:39.519796273 +0000 UTC m=+251.481305792"
Sep 10 14:15:40.417363 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:40.417330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:15:40.419717 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:40.419659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c4139c-2dde-4cd3-ba84-c9bee977f4ea-metrics-certs\") pod \"network-metrics-daemon-6ghsl\" (UID: \"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea\") " pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:15:40.629186 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:40.629158    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jzwbc\""
Sep 10 14:15:40.637417 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:40.637399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6ghsl"
Sep 10 14:15:40.903496 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:40.903326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6ghsl"]
Sep 10 14:15:40.905857 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:15:40.905832    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c4139c_2dde_4cd3_ba84_c9bee977f4ea.slice/crio-cfe0f06365a98bcfe975fc7baa4e2cca2d81d9c0ae14293220e54c2219793c78 WatchSource:0}: Error finding container cfe0f06365a98bcfe975fc7baa4e2cca2d81d9c0ae14293220e54c2219793c78: Status 404 returned error can't find the container with id cfe0f06365a98bcfe975fc7baa4e2cca2d81d9c0ae14293220e54c2219793c78
Sep 10 14:15:41.500112 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:41.500073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6ghsl" event={"ID":"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea","Type":"ContainerStarted","Data":"cfe0f06365a98bcfe975fc7baa4e2cca2d81d9c0ae14293220e54c2219793c78"}
Sep 10 14:15:42.850666 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:42.850625    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:15:43.509621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:43.509583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6ghsl" event={"ID":"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea","Type":"ContainerStarted","Data":"60a7e7b5c0fed6becf5cf4a0473c97a950725ce3135fa8d395c820dccf3c892c"}
Sep 10 14:15:43.509809 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:43.509645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6ghsl" event={"ID":"b9c4139c-2dde-4cd3-ba84-c9bee977f4ea","Type":"ContainerStarted","Data":"edc32ebcab764725c6624b043648f6cc421b64bf441552ee2f4891dcb059fc64"}
Sep 10 14:15:43.529357 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:15:43.529307    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6ghsl" podStartSLOduration=252.504216548 podStartE2EDuration="4m14.529291795s" totalImagesPullingTime="2.025075247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:15:43.526835578 +0000 UTC m=+255.488345095" watchObservedRunningTime="2026-09-10 14:15:43.529291795 +0000 UTC m=+255.490801313"
Sep 10 14:16:28.421572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:28.421539    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:16:28.423042 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:28.423018    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:16:28.425436 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:28.425416    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 14:16:37.850927 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:37.850890    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:16:37.866342 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:37.866318    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:16:38.724276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:16:38.724248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 14:18:22.866236 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.866200    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"]
Sep 10 14:18:22.869335 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.869317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:22.871830 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.871807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 14:18:22.872986 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.872969    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wsljf\""
Sep 10 14:18:22.873079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.872999    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 14:18:22.878770 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:22.878749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"]
Sep 10 14:18:23.001633 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.001600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.001819 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.001682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7sp5\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-kube-api-access-x7sp5\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.102143 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.102113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7sp5\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-kube-api-access-x7sp5\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.102143 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.102145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.110403 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.110375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.110516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.110428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7sp5\" (UniqueName: \"kubernetes.io/projected/7f894055-b77d-44d7-9e9c-2071417037ca-kube-api-access-x7sp5\") pod \"cert-manager-cainjector-559cdd8c87-wvzbs\" (UID: \"7f894055-b77d-44d7-9e9c-2071417037ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.178007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.177959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"
Sep 10 14:18:23.445848 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.445823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs"]
Sep 10 14:18:23.448264 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:18:23.448237    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f894055_b77d_44d7_9e9c_2071417037ca.slice/crio-f9fcaaaf40c1cecfe31abf340038f4fc628906c0124f3ffa5fb98988cc4d03f5 WatchSource:0}: Error finding container f9fcaaaf40c1cecfe31abf340038f4fc628906c0124f3ffa5fb98988cc4d03f5: Status 404 returned error can't find the container with id f9fcaaaf40c1cecfe31abf340038f4fc628906c0124f3ffa5fb98988cc4d03f5
Sep 10 14:18:23.450116 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:23.450097    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:18:24.092669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:24.092630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs" event={"ID":"7f894055-b77d-44d7-9e9c-2071417037ca","Type":"ContainerStarted","Data":"f9fcaaaf40c1cecfe31abf340038f4fc628906c0124f3ffa5fb98988cc4d03f5"}
Sep 10 14:18:27.112108 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:27.112056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs" event={"ID":"7f894055-b77d-44d7-9e9c-2071417037ca","Type":"ContainerStarted","Data":"5ead5ad2215c6c194219c4d0fbd4f2e8864cca1313224547116b83d905a0a37f"}
Sep 10 14:18:27.128501 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:27.128451    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wvzbs" podStartSLOduration=2.011681518 podStartE2EDuration="5.128435634s" totalImagesPullingTime="3.116754116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:18:27.127022094 +0000 UTC m=+419.088531619" watchObservedRunningTime="2026-09-10 14:18:27.128435634 +0000 UTC m=+419.089945151"
Sep 10 14:18:50.766738 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.766689    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"]
Sep 10 14:18:50.770138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.770122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.773711 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.773676    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:18:50.774512 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.774495    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 14:18:50.774915 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.774899    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 14:18:50.774977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.774933    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vtmhk\""
Sep 10 14:18:50.775244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.775222    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 14:18:50.778922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.778904    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 14:18:50.800457 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.800436    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"]
Sep 10 14:18:50.804769 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.804747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-manager-config\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.804947 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.804926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-metrics-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.805019 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.804986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.805079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.805046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkrfs\" (UniqueName: \"kubernetes.io/projected/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-kube-api-access-rkrfs\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.905622 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.905587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.905788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.905643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkrfs\" (UniqueName: \"kubernetes.io/projected/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-kube-api-access-rkrfs\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.905788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.905671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-manager-config\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.905788 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.905748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-metrics-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.906352 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.906332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-manager-config\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.908265 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.908241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-metrics-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.908364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.908245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-cert\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:50.915317 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:50.915267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkrfs\" (UniqueName: \"kubernetes.io/projected/d62a7918-bfdb-4bfc-bb18-0b66a31891c8-kube-api-access-rkrfs\") pod \"lws-controller-manager-7b64d8b6d6-vl696\" (UID: \"d62a7918-bfdb-4bfc-bb18-0b66a31891c8\") " pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:51.079531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:51.079457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:51.357799 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:51.357774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"]
Sep 10 14:18:51.360299 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:18:51.360265    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62a7918_bfdb_4bfc_bb18_0b66a31891c8.slice/crio-5ef07b69fb2fa57f30361dffceccdc93ef1e43546e62dfd1f30a373d7d50d5c0 WatchSource:0}: Error finding container 5ef07b69fb2fa57f30361dffceccdc93ef1e43546e62dfd1f30a373d7d50d5c0: Status 404 returned error can't find the container with id 5ef07b69fb2fa57f30361dffceccdc93ef1e43546e62dfd1f30a373d7d50d5c0
Sep 10 14:18:52.203680 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:52.203645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696" event={"ID":"d62a7918-bfdb-4bfc-bb18-0b66a31891c8","Type":"ContainerStarted","Data":"5ef07b69fb2fa57f30361dffceccdc93ef1e43546e62dfd1f30a373d7d50d5c0"}
Sep 10 14:18:55.194551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.194516    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"]
Sep 10 14:18:55.197947 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.197929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.200600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.200577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 14:18:55.200600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.200591    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 14:18:55.201184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.201167    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 14:18:55.201275 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.201261    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 14:18:55.201339 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.201265    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4t9x5\""
Sep 10 14:18:55.220138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.220108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696" event={"ID":"d62a7918-bfdb-4bfc-bb18-0b66a31891c8","Type":"ContainerStarted","Data":"0f9a7cbd2b1ee1532839be5ed9f6d283fcd3faaa9e9ebbd06e0e8bc4698e7322"}
Sep 10 14:18:55.220432 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.220413    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:18:55.221133 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.221101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"]
Sep 10 14:18:55.243119 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.243088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.243238 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.243124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.243238 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.243197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55lp9\" (UniqueName: \"kubernetes.io/projected/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-kube-api-access-55lp9\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.243400 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.243240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.243400 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.243299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.310196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.310138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696" podStartSLOduration=1.924403341 podStartE2EDuration="5.310118598s" totalImagesPullingTime="3.385715257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:18:55.309263378 +0000 UTC m=+447.270772897" watchObservedRunningTime="2026-09-10 14:18:55.310118598 +0000 UTC m=+447.271628148"
Sep 10 14:18:55.344753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.344723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.344753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.344754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.344977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.344781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55lp9\" (UniqueName: \"kubernetes.io/projected/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-kube-api-access-55lp9\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.344977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.344804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.344977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.344844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.345177 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.345155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.347246 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.347215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.347370 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.347324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.347370 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.347338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.354999 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.354977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55lp9\" (UniqueName: \"kubernetes.io/projected/70c591ef-cbaf-4bdf-8b23-3e92d8d51876-kube-api-access-55lp9\") pod \"opendatahub-operator-controller-manager-55f59f54dd-72wk2\" (UID: \"70c591ef-cbaf-4bdf-8b23-3e92d8d51876\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.507877 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.507807    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:18:55.791381 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:55.791340    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"]
Sep 10 14:18:55.795569 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:18:55.795537    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c591ef_cbaf_4bdf_8b23_3e92d8d51876.slice/crio-a604f45e5d7219dbf7df6ee4ffa98651b20184da4ea0df8bb40069b527d52be2 WatchSource:0}: Error finding container a604f45e5d7219dbf7df6ee4ffa98651b20184da4ea0df8bb40069b527d52be2: Status 404 returned error can't find the container with id a604f45e5d7219dbf7df6ee4ffa98651b20184da4ea0df8bb40069b527d52be2
Sep 10 14:18:56.225772 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:56.225674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2" event={"ID":"70c591ef-cbaf-4bdf-8b23-3e92d8d51876","Type":"ContainerStarted","Data":"a604f45e5d7219dbf7df6ee4ffa98651b20184da4ea0df8bb40069b527d52be2"}
Sep 10 14:18:59.242871 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:18:59.242798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2" event={"ID":"70c591ef-cbaf-4bdf-8b23-3e92d8d51876","Type":"ContainerStarted","Data":"ba616bcbbe29ac05b942ed02e75958ff577aada22fd74b20778797f5e7a4e140"}
Sep 10 14:19:00.248661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:00.248574    2581 generic.go:356] "Generic (PLEG): container finished" podID="70c591ef-cbaf-4bdf-8b23-3e92d8d51876" containerID="ba616bcbbe29ac05b942ed02e75958ff577aada22fd74b20778797f5e7a4e140" exitCode=0
Sep 10 14:19:00.249069 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:00.248662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2" event={"ID":"70c591ef-cbaf-4bdf-8b23-3e92d8d51876","Type":"ContainerDied","Data":"ba616bcbbe29ac05b942ed02e75958ff577aada22fd74b20778797f5e7a4e140"}
Sep 10 14:19:01.255200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:01.255167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2" event={"ID":"70c591ef-cbaf-4bdf-8b23-3e92d8d51876","Type":"ContainerStarted","Data":"c56f7255659098a4e8de9a58d7a230aec9fe633c0b4b1225df65a40089369b92"}
Sep 10 14:19:01.255600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:01.255284    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:19:01.276776 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:01.276733    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2" podStartSLOduration=1.951965438 podStartE2EDuration="6.276695209s" totalImagesPullingTime="3.297443886s" totalInitContainerRuntime="1.027285885s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:18:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:01.275202107 +0000 UTC m=+453.236711638" watchObservedRunningTime="2026-09-10 14:19:01.276695209 +0000 UTC m=+453.238204728"
Sep 10 14:19:06.227797 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:06.227765    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b64d8b6d6-vl696"
Sep 10 14:19:12.262983 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:12.262948    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-72wk2"
Sep 10 14:19:16.304395 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.304359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"]
Sep 10 14:19:16.310600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.310577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.313124 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.313096    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 14:19:16.313249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.313096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 14:19:16.313249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.313168    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 14:19:16.313249 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.313182    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 14:19:16.314428 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.314402    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kkbtb\""
Sep 10 14:19:16.320148 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.320127    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"]
Sep 10 14:19:16.413796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.413772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88rp8\" (UniqueName: \"kubernetes.io/projected/57186202-2dcf-40b5-bc92-a2d99fcb8771-kube-api-access-88rp8\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.413932 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.413827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57186202-2dcf-40b5-bc92-a2d99fcb8771-tmp\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.413932 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.413871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57186202-2dcf-40b5-bc92-a2d99fcb8771-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.515180 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.515152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57186202-2dcf-40b5-bc92-a2d99fcb8771-tmp\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.515307 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.515193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57186202-2dcf-40b5-bc92-a2d99fcb8771-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.515376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.515324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88rp8\" (UniqueName: \"kubernetes.io/projected/57186202-2dcf-40b5-bc92-a2d99fcb8771-kube-api-access-88rp8\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.517404 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.517369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57186202-2dcf-40b5-bc92-a2d99fcb8771-tmp\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.517580 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.517562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57186202-2dcf-40b5-bc92-a2d99fcb8771-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.522960 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.522939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88rp8\" (UniqueName: \"kubernetes.io/projected/57186202-2dcf-40b5-bc92-a2d99fcb8771-kube-api-access-88rp8\") pod \"kube-auth-proxy-85dcd64b9c-rctfp\" (UID: \"57186202-2dcf-40b5-bc92-a2d99fcb8771\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.623208 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.623147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"
Sep 10 14:19:16.897336 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:16.897263    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp"]
Sep 10 14:19:16.901638 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:19:16.901590    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57186202_2dcf_40b5_bc92_a2d99fcb8771.slice/crio-80ee7b858a620cd2b7e9af53b2333730f37da5227e7da3e5a0cd0fb3011a1b7f WatchSource:0}: Error finding container 80ee7b858a620cd2b7e9af53b2333730f37da5227e7da3e5a0cd0fb3011a1b7f: Status 404 returned error can't find the container with id 80ee7b858a620cd2b7e9af53b2333730f37da5227e7da3e5a0cd0fb3011a1b7f
Sep 10 14:19:17.326793 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:17.326751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp" event={"ID":"57186202-2dcf-40b5-bc92-a2d99fcb8771","Type":"ContainerStarted","Data":"80ee7b858a620cd2b7e9af53b2333730f37da5227e7da3e5a0cd0fb3011a1b7f"}
Sep 10 14:19:20.342288 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:20.342251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp" event={"ID":"57186202-2dcf-40b5-bc92-a2d99fcb8771","Type":"ContainerStarted","Data":"f37758f22529ac180180123334ac594a091bebd5825ab1953821d0273aea08dc"}
Sep 10 14:19:20.359931 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:19:20.359881    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-rctfp" podStartSLOduration=1.5302902459999999 podStartE2EDuration="4.359867045s" totalImagesPullingTime="2.829576799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:20.359337865 +0000 UTC m=+472.320847385" watchObservedRunningTime="2026-09-10 14:19:20.359867045 +0000 UTC m=+472.321376562"
Sep 10 14:21:14.164210 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.164176    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"]
Sep 10 14:21:14.167487 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.167468    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.172987 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.172953    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:21:14.173154 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.173132    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j42sg\""
Sep 10 14:21:14.173429 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.173406    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:21:14.200747 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.200711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"]
Sep 10 14:21:14.272459 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.272421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.272690 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.272583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2nj\" (UniqueName: \"kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.374031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.373992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2nj\" (UniqueName: \"kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.374031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.374033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.374395 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.374377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.384603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.384578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2nj\" (UniqueName: \"kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.477839 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.477745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.579655 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.579618    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"]
Sep 10 14:21:14.589366 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.589337    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"]
Sep 10 14:21:14.602656 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.602626    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:14.606175 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.606153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.626430 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.626399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:14.678080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.678044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2p5\" (UniqueName: \"kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.678259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.678168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.779439 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.779295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.779439 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.779351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2p5\" (UniqueName: \"kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.780051 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.779996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.791433 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.791398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2p5\" (UniqueName: \"kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-2lrnk\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:14.870677 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:14.870641    2581 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 10 14:21:14.870677 ip-10-0-131-209 kubenswrapper[2581]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-wzmqh_kuadrant-system_d6b60932-91fc-41ff-8cb2-28a399bf041b_0(3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-wzmqh to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e" Netns:"/var/run/netns/6a034e14-f36a-44b5-a1e4-6268cccaa5de" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-wzmqh;K8S_POD_INFRA_CONTAINER_ID=3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e;K8S_POD_UID=d6b60932-91fc-41ff-8cb2-28a399bf041b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh/d6b60932-91fc-41ff-8cb2-28a399bf041b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh 3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:1b [10.132.0.27/23]
Sep 10 14:21:14.870677 ip-10-0-131-209 kubenswrapper[2581]:         '
Sep 10 14:21:14.870677 ip-10-0-131-209 kubenswrapper[2581]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 14:21:14.870677 ip-10-0-131-209 kubenswrapper[2581]:  >
Sep 10 14:21:14.870966 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:14.870716    2581 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 10 14:21:14.870966 ip-10-0-131-209 kubenswrapper[2581]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-wzmqh_kuadrant-system_d6b60932-91fc-41ff-8cb2-28a399bf041b_0(3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-wzmqh to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e" Netns:"/var/run/netns/6a034e14-f36a-44b5-a1e4-6268cccaa5de" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-wzmqh;K8S_POD_INFRA_CONTAINER_ID=3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e;K8S_POD_UID=d6b60932-91fc-41ff-8cb2-28a399bf041b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh/d6b60932-91fc-41ff-8cb2-28a399bf041b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh 3ac969ce883014c155266fcbd7b6b7b435c20d905bdc46c15a424f8d9f8a0c2e network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:1b [10.132.0.27/23]
Sep 10 14:21:14.870966 ip-10-0-131-209 kubenswrapper[2581]:         '
Sep 10 14:21:14.870966 ip-10-0-131-209 kubenswrapper[2581]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 14:21:14.870966 ip-10-0-131-209 kubenswrapper[2581]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:14.918527 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:14.918487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:15.215085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.215059    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:15.217851 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:21:15.217826    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9a915b_5a41_45a3_ac73_a4ed9c173a09.slice/crio-2140b98bd43371e60d37ad7044d8d7b3d9d4cfc3242a41f2915df10be6d065f5 WatchSource:0}: Error finding container 2140b98bd43371e60d37ad7044d8d7b3d9d4cfc3242a41f2915df10be6d065f5: Status 404 returned error can't find the container with id 2140b98bd43371e60d37ad7044d8d7b3d9d4cfc3242a41f2915df10be6d065f5
Sep 10 14:21:15.826918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.826888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:15.826918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.826900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" event={"ID":"1b9a915b-5a41-45a3-ac73-a4ed9c173a09","Type":"ContainerStarted","Data":"2140b98bd43371e60d37ad7044d8d7b3d9d4cfc3242a41f2915df10be6d065f5"}
Sep 10 14:21:15.829525 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:15.829495    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" is forbidden: User \"system:node:ip-10-0-131-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-209.ec2.internal' and this object" podUID="d6b60932-91fc-41ff-8cb2-28a399bf041b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:15.835385 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.835359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:15.837743 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:15.837720    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" is forbidden: User \"system:node:ip-10-0-131-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-209.ec2.internal' and this object" podUID="d6b60932-91fc-41ff-8cb2-28a399bf041b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:15.990861 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.990824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw2nj\" (UniqueName: \"kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj\") pod \"d6b60932-91fc-41ff-8cb2-28a399bf041b\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") "
Sep 10 14:21:15.991045 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.990892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume\") pod \"d6b60932-91fc-41ff-8cb2-28a399bf041b\" (UID: \"d6b60932-91fc-41ff-8cb2-28a399bf041b\") "
Sep 10 14:21:15.991227 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.991179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d6b60932-91fc-41ff-8cb2-28a399bf041b" (UID: "d6b60932-91fc-41ff-8cb2-28a399bf041b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:21:15.993793 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:15.993763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj" (OuterVolumeSpecName: "kube-api-access-fw2nj") pod "d6b60932-91fc-41ff-8cb2-28a399bf041b" (UID: "d6b60932-91fc-41ff-8cb2-28a399bf041b"). InnerVolumeSpecName "kube-api-access-fw2nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:21:16.092242 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:16.092129    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw2nj\" (UniqueName: \"kubernetes.io/projected/d6b60932-91fc-41ff-8cb2-28a399bf041b-kube-api-access-fw2nj\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:21:16.092242 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:16.092166    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6b60932-91fc-41ff-8cb2-28a399bf041b-extensions-socket-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:21:16.533850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:16.533800    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6b60932-91fc-41ff-8cb2-28a399bf041b" path="/var/lib/kubelet/pods/d6b60932-91fc-41ff-8cb2-28a399bf041b/volumes"
Sep 10 14:21:16.833128 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:16.833039    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:16.838470 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:16.838430    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" is forbidden: User \"system:node:ip-10-0-131-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-209.ec2.internal' and this object" podUID="d6b60932-91fc-41ff-8cb2-28a399bf041b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:18.530491 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:18.530442    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-wzmqh\" is forbidden: User \"system:node:ip-10-0-131-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-209.ec2.internal' and this object" podUID="d6b60932-91fc-41ff-8cb2-28a399bf041b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-wzmqh"
Sep 10 14:21:19.850032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:19.849993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" event={"ID":"1b9a915b-5a41-45a3-ac73-a4ed9c173a09","Type":"ContainerStarted","Data":"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"}
Sep 10 14:21:19.850431 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:19.850114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:19.871724 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:19.871654    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" podStartSLOduration=2.09131226 podStartE2EDuration="5.871636637s" totalImagesPullingTime="3.780324377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:19.869977194 +0000 UTC m=+591.831486736" watchObservedRunningTime="2026-09-10 14:21:19.871636637 +0000 UTC m=+591.833146157"
Sep 10 14:21:28.454502 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:28.454476    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:21:28.454923 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:28.454479    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:21:30.857194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:30.857163    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:43.677331 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.677294    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:43.679073 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.677534    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" podUID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" containerName="manager" containerID="cri-o://13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f" gracePeriod=10
Sep 10 14:21:43.925217 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.925189    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:43.955037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.954870    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" containerID="13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f" exitCode=0
Sep 10 14:21:43.955037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.954925    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"
Sep 10 14:21:43.955037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.954961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" event={"ID":"1b9a915b-5a41-45a3-ac73-a4ed9c173a09","Type":"ContainerDied","Data":"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"}
Sep 10 14:21:43.955037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.955025    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk" event={"ID":"1b9a915b-5a41-45a3-ac73-a4ed9c173a09","Type":"ContainerDied","Data":"2140b98bd43371e60d37ad7044d8d7b3d9d4cfc3242a41f2915df10be6d065f5"}
Sep 10 14:21:43.955350 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.955056    2581 scope.go:117] "RemoveContainer" containerID="13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"
Sep 10 14:21:43.964181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.964153    2581 scope.go:117] "RemoveContainer" containerID="13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"
Sep 10 14:21:43.964474 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:21:43.964452    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f\": container with ID starting with 13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f not found: ID does not exist" containerID="13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"
Sep 10 14:21:43.964564 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:43.964483    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f"} err="failed to get container status \"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f\": rpc error: code = NotFound desc = could not find container \"13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f\": container with ID starting with 13f56cf0e49651781163bd27b900d18af85b02fdc2e0c7406bf6e2e4b2a2811f not found: ID does not exist"
Sep 10 14:21:44.041010 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.040970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq2p5\" (UniqueName: \"kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5\") pod \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") "
Sep 10 14:21:44.041181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.041031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume\") pod \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\" (UID: \"1b9a915b-5a41-45a3-ac73-a4ed9c173a09\") "
Sep 10 14:21:44.041405 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.041377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b9a915b-5a41-45a3-ac73-a4ed9c173a09" (UID: "1b9a915b-5a41-45a3-ac73-a4ed9c173a09"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:21:44.043198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.043179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5" (OuterVolumeSpecName: "kube-api-access-jq2p5") pod "1b9a915b-5a41-45a3-ac73-a4ed9c173a09" (UID: "1b9a915b-5a41-45a3-ac73-a4ed9c173a09"). InnerVolumeSpecName "kube-api-access-jq2p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:21:44.142551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.142513    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2p5\" (UniqueName: \"kubernetes.io/projected/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-kube-api-access-jq2p5\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:21:44.142551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.142543    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b9a915b-5a41-45a3-ac73-a4ed9c173a09-extensions-socket-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:21:44.281839 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.281805    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:44.286135 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.286105    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-2lrnk"]
Sep 10 14:21:44.530922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:21:44.530841    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" path="/var/lib/kubelet/pods/1b9a915b-5a41-45a3-ac73-a4ed9c173a09/volumes"
Sep 10 14:22:01.916189 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.916099    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:01.916651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.916598    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" containerName="manager"
Sep 10 14:22:01.916651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.916615    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" containerName="manager"
Sep 10 14:22:01.916750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.916725    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9a915b-5a41-45a3-ac73-a4ed9c173a09" containerName="manager"
Sep 10 14:22:01.918928 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.918911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:01.921610 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.921589    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 14:22:01.921795 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.921774    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:22:01.921918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.921802    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:22:01.922647 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.922633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gz7vr\""
Sep 10 14:22:01.926406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:01.926367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:02.000810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.000759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.000999 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.000865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58kr\" (UniqueName: \"kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.018481 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.018124    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:02.101498 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.101463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.101672 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.101525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58kr\" (UniqueName: \"kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.102126 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.102104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.110200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.110169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58kr\" (UniqueName: \"kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr\") pod \"limitador-limitador-8588749977-pk9zv\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") " pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.230629 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.230535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:02.500463 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.500402    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:02.502775 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:02.502748    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd77d4c46_a995_4001_bc40_7dc85d0a9c95.slice/crio-ff5ded2b880c4d57800b3749c97b848e9412147ea8eb54c7050ac0531849ee33 WatchSource:0}: Error finding container ff5ded2b880c4d57800b3749c97b848e9412147ea8eb54c7050ac0531849ee33: Status 404 returned error can't find the container with id ff5ded2b880c4d57800b3749c97b848e9412147ea8eb54c7050ac0531849ee33
Sep 10 14:22:02.708599 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.708562    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:02.711443 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.711427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:02.714084 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.714065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zp994\""
Sep 10 14:22:02.719525 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.719494    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:02.806683 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.806652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsmmg\" (UniqueName: \"kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg\") pod \"authorino-7bcfd8f9cd-2fdxl\" (UID: \"af0be998-de98-401c-bc97-e7392cf1a3f9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:02.839688 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.839650    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:02.842070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.842047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:02.850608 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.850581    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:02.907282 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.907248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsmmg\" (UniqueName: \"kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg\") pod \"authorino-7bcfd8f9cd-2fdxl\" (UID: \"af0be998-de98-401c-bc97-e7392cf1a3f9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:02.907434 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.907296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xr59\" (UniqueName: \"kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59\") pod \"authorino-59c974bd48-q6frz\" (UID: \"71504590-6523-48f2-b66d-83e468dcd6e7\") " pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:02.915123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:02.915091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsmmg\" (UniqueName: \"kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg\") pod \"authorino-7bcfd8f9cd-2fdxl\" (UID: \"af0be998-de98-401c-bc97-e7392cf1a3f9\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:03.008360 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.008318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xr59\" (UniqueName: \"kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59\") pod \"authorino-59c974bd48-q6frz\" (UID: \"71504590-6523-48f2-b66d-83e468dcd6e7\") " pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:03.017368 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.017344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xr59\" (UniqueName: \"kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59\") pod \"authorino-59c974bd48-q6frz\" (UID: \"71504590-6523-48f2-b66d-83e468dcd6e7\") " pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:03.021085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.021068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:03.038637 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.038609    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" event={"ID":"d77d4c46-a995-4001-bc40-7dc85d0a9c95","Type":"ContainerStarted","Data":"ff5ded2b880c4d57800b3749c97b848e9412147ea8eb54c7050ac0531849ee33"}
Sep 10 14:22:03.151135 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.151105    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:03.291185 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.291161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:03.293606 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:03.293576    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf0be998_de98_401c_bc97_e7392cf1a3f9.slice/crio-75c7ed8fe694cb6441095e7a571c48090d9c533defa7e897ffa110a4d51e0df1 WatchSource:0}: Error finding container 75c7ed8fe694cb6441095e7a571c48090d9c533defa7e897ffa110a4d51e0df1: Status 404 returned error can't find the container with id 75c7ed8fe694cb6441095e7a571c48090d9c533defa7e897ffa110a4d51e0df1
Sep 10 14:22:03.418843 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:03.418767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:03.422488 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:03.422457    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71504590_6523_48f2_b66d_83e468dcd6e7.slice/crio-7c753dde7614eaa16b3114d6fd860c872303a1ed7741ac66c013b5b1e2ee1d34 WatchSource:0}: Error finding container 7c753dde7614eaa16b3114d6fd860c872303a1ed7741ac66c013b5b1e2ee1d34: Status 404 returned error can't find the container with id 7c753dde7614eaa16b3114d6fd860c872303a1ed7741ac66c013b5b1e2ee1d34
Sep 10 14:22:04.047626 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:04.047577    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6frz" event={"ID":"71504590-6523-48f2-b66d-83e468dcd6e7","Type":"ContainerStarted","Data":"7c753dde7614eaa16b3114d6fd860c872303a1ed7741ac66c013b5b1e2ee1d34"}
Sep 10 14:22:04.049160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:04.049132    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" event={"ID":"af0be998-de98-401c-bc97-e7392cf1a3f9","Type":"ContainerStarted","Data":"75c7ed8fe694cb6441095e7a571c48090d9c533defa7e897ffa110a4d51e0df1"}
Sep 10 14:22:05.055304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:05.055263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" event={"ID":"d77d4c46-a995-4001-bc40-7dc85d0a9c95","Type":"ContainerStarted","Data":"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"}
Sep 10 14:22:05.055808 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:05.055376    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:05.071889 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:05.071834    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" podStartSLOduration=2.309382427 podStartE2EDuration="4.071819279s" totalImagesPullingTime="1.762436852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:05.070779202 +0000 UTC m=+637.032288713" watchObservedRunningTime="2026-09-10 14:22:05.071819279 +0000 UTC m=+637.033328796"
Sep 10 14:22:06.061462 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:06.061423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6frz" event={"ID":"71504590-6523-48f2-b66d-83e468dcd6e7","Type":"ContainerStarted","Data":"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"}
Sep 10 14:22:06.063011 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:06.062970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" event={"ID":"af0be998-de98-401c-bc97-e7392cf1a3f9","Type":"ContainerStarted","Data":"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"}
Sep 10 14:22:06.076768 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:06.076722    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-q6frz" podStartSLOduration=2.145671281 podStartE2EDuration="4.07668038s" totalImagesPullingTime="1.931009099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:06.076117218 +0000 UTC m=+638.037626736" watchObservedRunningTime="2026-09-10 14:22:06.07668038 +0000 UTC m=+638.038189898"
Sep 10 14:22:06.091996 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:06.091934    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" podStartSLOduration=2.027737941 podStartE2EDuration="4.091914417s" totalImagesPullingTime="2.064176476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:06.089683272 +0000 UTC m=+638.051192797" watchObservedRunningTime="2026-09-10 14:22:06.091914417 +0000 UTC m=+638.053423936"
Sep 10 14:22:06.102163 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:06.102129    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:08.071658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:08.071614    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" podUID="af0be998-de98-401c-bc97-e7392cf1a3f9" containerName="authorino" containerID="cri-o://2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025" gracePeriod=30
Sep 10 14:22:08.319313 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:08.319288    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:08.459157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:08.459062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsmmg\" (UniqueName: \"kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg\") pod \"af0be998-de98-401c-bc97-e7392cf1a3f9\" (UID: \"af0be998-de98-401c-bc97-e7392cf1a3f9\") "
Sep 10 14:22:08.461146 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:08.461114    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg" (OuterVolumeSpecName: "kube-api-access-rsmmg") pod "af0be998-de98-401c-bc97-e7392cf1a3f9" (UID: "af0be998-de98-401c-bc97-e7392cf1a3f9"). InnerVolumeSpecName "kube-api-access-rsmmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:08.559837 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:08.559800    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsmmg\" (UniqueName: \"kubernetes.io/projected/af0be998-de98-401c-bc97-e7392cf1a3f9-kube-api-access-rsmmg\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:09.076758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.076718    2581 generic.go:356] "Generic (PLEG): container finished" podID="af0be998-de98-401c-bc97-e7392cf1a3f9" containerID="2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025" exitCode=0
Sep 10 14:22:09.076758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.076752    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl"
Sep 10 14:22:09.076758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.076762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" event={"ID":"af0be998-de98-401c-bc97-e7392cf1a3f9","Type":"ContainerDied","Data":"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"}
Sep 10 14:22:09.077273 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.076795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2fdxl" event={"ID":"af0be998-de98-401c-bc97-e7392cf1a3f9","Type":"ContainerDied","Data":"75c7ed8fe694cb6441095e7a571c48090d9c533defa7e897ffa110a4d51e0df1"}
Sep 10 14:22:09.077273 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.076813    2581 scope.go:117] "RemoveContainer" containerID="2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"
Sep 10 14:22:09.085056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.085037    2581 scope.go:117] "RemoveContainer" containerID="2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"
Sep 10 14:22:09.085340 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:09.085316    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025\": container with ID starting with 2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025 not found: ID does not exist" containerID="2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"
Sep 10 14:22:09.085385 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.085350    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025"} err="failed to get container status \"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025\": rpc error: code = NotFound desc = could not find container \"2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025\": container with ID starting with 2fa138629b7680642661b7340ab4226cb8e8f32b0c3ff5a949999f78910d2025 not found: ID does not exist"
Sep 10 14:22:09.092919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.092893    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:09.098978 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:09.098954    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2fdxl"]
Sep 10 14:22:10.530208 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:10.530175    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0be998-de98-401c-bc97-e7392cf1a3f9" path="/var/lib/kubelet/pods/af0be998-de98-401c-bc97-e7392cf1a3f9/volumes"
Sep 10 14:22:15.828201 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:15.828159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:15.828756 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:15.828360    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" podUID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" containerName="limitador" containerID="cri-o://46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332" gracePeriod=30
Sep 10 14:22:15.829178 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:15.829073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:16.377918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.377892    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:16.523842 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.523815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m58kr\" (UniqueName: \"kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr\") pod \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") "
Sep 10 14:22:16.524016 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.523920    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file\") pod \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\" (UID: \"d77d4c46-a995-4001-bc40-7dc85d0a9c95\") "
Sep 10 14:22:16.524276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.524255    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file" (OuterVolumeSpecName: "config-file") pod "d77d4c46-a995-4001-bc40-7dc85d0a9c95" (UID: "d77d4c46-a995-4001-bc40-7dc85d0a9c95"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:22:16.525967 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.525942    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr" (OuterVolumeSpecName: "kube-api-access-m58kr") pod "d77d4c46-a995-4001-bc40-7dc85d0a9c95" (UID: "d77d4c46-a995-4001-bc40-7dc85d0a9c95"). InnerVolumeSpecName "kube-api-access-m58kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:16.625244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.625212    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d77d4c46-a995-4001-bc40-7dc85d0a9c95-config-file\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:16.625244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:16.625241    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m58kr\" (UniqueName: \"kubernetes.io/projected/d77d4c46-a995-4001-bc40-7dc85d0a9c95-kube-api-access-m58kr\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:17.114190 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.114155    2581 generic.go:356] "Generic (PLEG): container finished" podID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" containerID="46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332" exitCode=0
Sep 10 14:22:17.114638 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.114200    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv"
Sep 10 14:22:17.114638 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.114225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" event={"ID":"d77d4c46-a995-4001-bc40-7dc85d0a9c95","Type":"ContainerDied","Data":"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"}
Sep 10 14:22:17.114638 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.114268    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pk9zv" event={"ID":"d77d4c46-a995-4001-bc40-7dc85d0a9c95","Type":"ContainerDied","Data":"ff5ded2b880c4d57800b3749c97b848e9412147ea8eb54c7050ac0531849ee33"}
Sep 10 14:22:17.114638 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.114293    2581 scope.go:117] "RemoveContainer" containerID="46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"
Sep 10 14:22:17.124480 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.124416    2581 scope.go:117] "RemoveContainer" containerID="46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"
Sep 10 14:22:17.125089 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:17.125061    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332\": container with ID starting with 46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332 not found: ID does not exist" containerID="46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"
Sep 10 14:22:17.125186 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.125105    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332"} err="failed to get container status \"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332\": rpc error: code = NotFound desc = could not find container \"46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332\": container with ID starting with 46f5e05aa69e5f96038b2a3315479e20b23c279450e39340766c014f12ed3332 not found: ID does not exist"
Sep 10 14:22:17.138212 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.138184    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:17.142518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:17.142498    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pk9zv"]
Sep 10 14:22:18.530861 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:18.530821    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" path="/var/lib/kubelet/pods/d77d4c46-a995-4001-bc40-7dc85d0a9c95/volumes"
Sep 10 14:22:20.964978 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.964945    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"]
Sep 10 14:22:20.965359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965276    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" containerName="limitador"
Sep 10 14:22:20.965359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965287    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" containerName="limitador"
Sep 10 14:22:20.965359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965300    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0be998-de98-401c-bc97-e7392cf1a3f9" containerName="authorino"
Sep 10 14:22:20.965359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965306    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0be998-de98-401c-bc97-e7392cf1a3f9" containerName="authorino"
Sep 10 14:22:20.965489 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965363    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="af0be998-de98-401c-bc97-e7392cf1a3f9" containerName="authorino"
Sep 10 14:22:20.965489 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.965370    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d77d4c46-a995-4001-bc40-7dc85d0a9c95" containerName="limitador"
Sep 10 14:22:20.968207 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.968188    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:20.970968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.970944    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:22:20.970968 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.970959    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:22:20.971878 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.971858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-rrgmb\""
Sep 10 14:22:20.971951 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.971860    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 14:22:20.978830 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:20.978810    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"]
Sep 10 14:22:21.058052 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.058015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxm5x\" (UniqueName: \"kubernetes.io/projected/4f8c8d38-afbe-478c-84a0-432a4cb1647f-kube-api-access-dxm5x\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.058202 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.058093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8d38-afbe-478c-84a0-432a4cb1647f-data\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.159471 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.159436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8d38-afbe-478c-84a0-432a4cb1647f-data\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.159650 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.159478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxm5x\" (UniqueName: \"kubernetes.io/projected/4f8c8d38-afbe-478c-84a0-432a4cb1647f-kube-api-access-dxm5x\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.159832 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.159812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8d38-afbe-478c-84a0-432a4cb1647f-data\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.171691 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.171667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxm5x\" (UniqueName: \"kubernetes.io/projected/4f8c8d38-afbe-478c-84a0-432a4cb1647f-kube-api-access-dxm5x\") pod \"postgres-69fb4594cf-bf7sq\" (UID: \"4f8c8d38-afbe-478c-84a0-432a4cb1647f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.279762 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.279731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:21.547454 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:21.547298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"]
Sep 10 14:22:21.550135 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:21.550109    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8c8d38_afbe_478c_84a0_432a4cb1647f.slice/crio-6e3b94bc6dbe053001275d0411881388c814c2205fc69382800d95c5fd720f20 WatchSource:0}: Error finding container 6e3b94bc6dbe053001275d0411881388c814c2205fc69382800d95c5fd720f20: Status 404 returned error can't find the container with id 6e3b94bc6dbe053001275d0411881388c814c2205fc69382800d95c5fd720f20
Sep 10 14:22:22.139473 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:22.139437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq" event={"ID":"4f8c8d38-afbe-478c-84a0-432a4cb1647f","Type":"ContainerStarted","Data":"6e3b94bc6dbe053001275d0411881388c814c2205fc69382800d95c5fd720f20"}
Sep 10 14:22:27.164671 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:27.164630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq" event={"ID":"4f8c8d38-afbe-478c-84a0-432a4cb1647f","Type":"ContainerStarted","Data":"dcae77b1908d9eb75467a5ea4b2ae32a688119e6516346927694bd8cee7ae398"}
Sep 10 14:22:27.165173 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:27.164747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:27.181219 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:27.181152    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq" podStartSLOduration=2.5589899259999997 podStartE2EDuration="7.18113551s" totalImagesPullingTime="4.622145584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:27.179518616 +0000 UTC m=+659.141028157" watchObservedRunningTime="2026-09-10 14:22:27.18113551 +0000 UTC m=+659.142645029"
Sep 10 14:22:33.198520 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.198486    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf7sq"
Sep 10 14:22:33.731015 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.730985    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:33.734840 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.734820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:33.740637 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.740612    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:33.767518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.767489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd4j9\" (UniqueName: \"kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9\") pod \"authorino-557f876f69-pfglj\" (UID: \"2ad364ae-4d6a-4553-b426-d4d38dbbccf5\") " pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:33.868466 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.868428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd4j9\" (UniqueName: \"kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9\") pod \"authorino-557f876f69-pfglj\" (UID: \"2ad364ae-4d6a-4553-b426-d4d38dbbccf5\") " pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:33.876602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.876569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd4j9\" (UniqueName: \"kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9\") pod \"authorino-557f876f69-pfglj\" (UID: \"2ad364ae-4d6a-4553-b426-d4d38dbbccf5\") " pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:33.953467 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.953431    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:33.953682 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.953671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:33.977096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.977067    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:33.979453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.979431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:33.986997 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:33.986968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:34.070753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.070722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzzvm\" (UniqueName: \"kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm\") pod \"authorino-7896445869-kthzw\" (UID: \"336ccf94-7fee-4ac2-975a-486e4b633e6f\") " pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:34.171158 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.171119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzzvm\" (UniqueName: \"kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm\") pod \"authorino-7896445869-kthzw\" (UID: \"336ccf94-7fee-4ac2-975a-486e4b633e6f\") " pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:34.180260 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.180237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzzvm\" (UniqueName: \"kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm\") pod \"authorino-7896445869-kthzw\" (UID: \"336ccf94-7fee-4ac2-975a-486e4b633e6f\") " pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:34.224589 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.224552    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:34.227920 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:34.227884    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ad364ae_4d6a_4553_b426_d4d38dbbccf5.slice/crio-a2572844c1185b05f7f61e734b2971eff3e15f77b3d12073c67b5316a32b2ab9 WatchSource:0}: Error finding container a2572844c1185b05f7f61e734b2971eff3e15f77b3d12073c67b5316a32b2ab9: Status 404 returned error can't find the container with id a2572844c1185b05f7f61e734b2971eff3e15f77b3d12073c67b5316a32b2ab9
Sep 10 14:22:34.289798 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.289764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:34.555674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:34.555647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:34.558013 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:34.557980    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod336ccf94_7fee_4ac2_975a_486e4b633e6f.slice/crio-9162f22d44a8cf215925a76ca3bde8ba08f19371b2ee2001c1433a46d0d1759a WatchSource:0}: Error finding container 9162f22d44a8cf215925a76ca3bde8ba08f19371b2ee2001c1433a46d0d1759a: Status 404 returned error can't find the container with id 9162f22d44a8cf215925a76ca3bde8ba08f19371b2ee2001c1433a46d0d1759a
Sep 10 14:22:35.199857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:35.199817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pfglj" event={"ID":"2ad364ae-4d6a-4553-b426-d4d38dbbccf5","Type":"ContainerStarted","Data":"a2572844c1185b05f7f61e734b2971eff3e15f77b3d12073c67b5316a32b2ab9"}
Sep 10 14:22:35.200940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:35.200917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7896445869-kthzw" event={"ID":"336ccf94-7fee-4ac2-975a-486e4b633e6f","Type":"ContainerStarted","Data":"9162f22d44a8cf215925a76ca3bde8ba08f19371b2ee2001c1433a46d0d1759a"}
Sep 10 14:22:36.207332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.207292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pfglj" event={"ID":"2ad364ae-4d6a-4553-b426-d4d38dbbccf5","Type":"ContainerStarted","Data":"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"}
Sep 10 14:22:36.207332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.207326    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-pfglj" podUID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" containerName="authorino" containerID="cri-o://32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819" gracePeriod=30
Sep 10 14:22:36.209079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.209054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7896445869-kthzw" event={"ID":"336ccf94-7fee-4ac2-975a-486e4b633e6f","Type":"ContainerStarted","Data":"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"}
Sep 10 14:22:36.223867 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.223777    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-pfglj" podStartSLOduration=1.4690139420000001 podStartE2EDuration="3.223763288s" totalImagesPullingTime="1.754749346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:36.223231861 +0000 UTC m=+668.184741380" watchObservedRunningTime="2026-09-10 14:22:36.223763288 +0000 UTC m=+668.185272806"
Sep 10 14:22:36.241036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.240995    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7896445869-kthzw" podStartSLOduration=2.408539787 podStartE2EDuration="3.240980905s" totalImagesPullingTime="832.441118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:36.239526469 +0000 UTC m=+668.201035985" watchObservedRunningTime="2026-09-10 14:22:36.240980905 +0000 UTC m=+668.202490423"
Sep 10 14:22:36.266157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.266126    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:36.266316 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.266297    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-q6frz" podUID="71504590-6523-48f2-b66d-83e468dcd6e7" containerName="authorino" containerID="cri-o://2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a" gracePeriod=30
Sep 10 14:22:36.360475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.360436    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:22:36.373732 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.373684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:22:36.373863 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.373809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.376409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.376388    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 14:22:36.492359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.492299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdtt\" (UniqueName: \"kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.492534 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.492373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.581423 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.581394    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:36.593138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.593112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdtt\" (UniqueName: \"kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.593254 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.593164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.595502 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.595476    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.599898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.599879    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:36.600632 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.600615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdtt\" (UniqueName: \"kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt\") pod \"authorino-b7879d44c-mpt2s\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") " pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.684893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.684857    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:22:36.693688 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.693662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xr59\" (UniqueName: \"kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59\") pod \"71504590-6523-48f2-b66d-83e468dcd6e7\" (UID: \"71504590-6523-48f2-b66d-83e468dcd6e7\") "
Sep 10 14:22:36.693835 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.693738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd4j9\" (UniqueName: \"kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9\") pod \"2ad364ae-4d6a-4553-b426-d4d38dbbccf5\" (UID: \"2ad364ae-4d6a-4553-b426-d4d38dbbccf5\") "
Sep 10 14:22:36.695635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.695607    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9" (OuterVolumeSpecName: "kube-api-access-zd4j9") pod "2ad364ae-4d6a-4553-b426-d4d38dbbccf5" (UID: "2ad364ae-4d6a-4553-b426-d4d38dbbccf5"). InnerVolumeSpecName "kube-api-access-zd4j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:36.695755 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.695638    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59" (OuterVolumeSpecName: "kube-api-access-6xr59") pod "71504590-6523-48f2-b66d-83e468dcd6e7" (UID: "71504590-6523-48f2-b66d-83e468dcd6e7"). InnerVolumeSpecName "kube-api-access-6xr59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:36.794585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.794556    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd4j9\" (UniqueName: \"kubernetes.io/projected/2ad364ae-4d6a-4553-b426-d4d38dbbccf5-kube-api-access-zd4j9\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:36.794585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.794584    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xr59\" (UniqueName: \"kubernetes.io/projected/71504590-6523-48f2-b66d-83e468dcd6e7-kube-api-access-6xr59\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:36.993859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:36.993833    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:22:36.996048 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:36.996023    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod784c7c62_511c_4785_ab45_7c67f2ea5677.slice/crio-18bde5750bb892f6d905e51db1f490baed8c153cc69e19da8b8e425f2c334c16 WatchSource:0}: Error finding container 18bde5750bb892f6d905e51db1f490baed8c153cc69e19da8b8e425f2c334c16: Status 404 returned error can't find the container with id 18bde5750bb892f6d905e51db1f490baed8c153cc69e19da8b8e425f2c334c16
Sep 10 14:22:37.215167 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.215088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b7879d44c-mpt2s" event={"ID":"784c7c62-511c-4785-ab45-7c67f2ea5677","Type":"ContainerStarted","Data":"18bde5750bb892f6d905e51db1f490baed8c153cc69e19da8b8e425f2c334c16"}
Sep 10 14:22:37.216406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.216378    2581 generic.go:356] "Generic (PLEG): container finished" podID="71504590-6523-48f2-b66d-83e468dcd6e7" containerID="2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a" exitCode=0
Sep 10 14:22:37.216533 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.216422    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6frz"
Sep 10 14:22:37.216533 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.216434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6frz" event={"ID":"71504590-6523-48f2-b66d-83e468dcd6e7","Type":"ContainerDied","Data":"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"}
Sep 10 14:22:37.216533 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.216472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6frz" event={"ID":"71504590-6523-48f2-b66d-83e468dcd6e7","Type":"ContainerDied","Data":"7c753dde7614eaa16b3114d6fd860c872303a1ed7741ac66c013b5b1e2ee1d34"}
Sep 10 14:22:37.216533 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.216488    2581 scope.go:117] "RemoveContainer" containerID="2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"
Sep 10 14:22:37.224387 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.224357    2581 generic.go:356] "Generic (PLEG): container finished" podID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" containerID="32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819" exitCode=0
Sep 10 14:22:37.224515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.224431    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pfglj"
Sep 10 14:22:37.224515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.224440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pfglj" event={"ID":"2ad364ae-4d6a-4553-b426-d4d38dbbccf5","Type":"ContainerDied","Data":"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"}
Sep 10 14:22:37.224515 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.224494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pfglj" event={"ID":"2ad364ae-4d6a-4553-b426-d4d38dbbccf5","Type":"ContainerDied","Data":"a2572844c1185b05f7f61e734b2971eff3e15f77b3d12073c67b5316a32b2ab9"}
Sep 10 14:22:37.232160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.232137    2581 scope.go:117] "RemoveContainer" containerID="2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"
Sep 10 14:22:37.232449 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:37.232426    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a\": container with ID starting with 2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a not found: ID does not exist" containerID="2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"
Sep 10 14:22:37.232505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.232460    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a"} err="failed to get container status \"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a\": rpc error: code = NotFound desc = could not find container \"2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a\": container with ID starting with 2b7c6a293d10618c56501983ffbb2842ce1f956bf530603e677ce95612ba968a not found: ID does not exist"
Sep 10 14:22:37.232505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.232481    2581 scope.go:117] "RemoveContainer" containerID="32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"
Sep 10 14:22:37.240313 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.240295    2581 scope.go:117] "RemoveContainer" containerID="32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"
Sep 10 14:22:37.240548 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:37.240533    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819\": container with ID starting with 32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819 not found: ID does not exist" containerID="32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"
Sep 10 14:22:37.240609 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.240553    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819"} err="failed to get container status \"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819\": rpc error: code = NotFound desc = could not find container \"32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819\": container with ID starting with 32099ae794673f3e8feda142a803ad53a861639e8c33dd2fead942dd484e6819 not found: ID does not exist"
Sep 10 14:22:37.249002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.248978    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:37.252391 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.252367    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6frz"]
Sep 10 14:22:37.262997 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.262974    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:37.267102 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:37.267082    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-pfglj"]
Sep 10 14:22:38.231539 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.231440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b7879d44c-mpt2s" event={"ID":"784c7c62-511c-4785-ab45-7c67f2ea5677","Type":"ContainerStarted","Data":"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"}
Sep 10 14:22:38.247455 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.247400    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b7879d44c-mpt2s" podStartSLOduration=1.363635019 podStartE2EDuration="2.247386901s" totalImagesPullingTime="883.751882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:38.245778662 +0000 UTC m=+670.207288181" watchObservedRunningTime="2026-09-10 14:22:38.247386901 +0000 UTC m=+670.208896419"
Sep 10 14:22:38.271828 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.271792    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:38.272021 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.271956    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7896445869-kthzw" podUID="336ccf94-7fee-4ac2-975a-486e4b633e6f" containerName="authorino" containerID="cri-o://ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655" gracePeriod=30
Sep 10 14:22:38.516518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.516492    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:38.533425 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.533396    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" path="/var/lib/kubelet/pods/2ad364ae-4d6a-4553-b426-d4d38dbbccf5/volumes"
Sep 10 14:22:38.533753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.533740    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71504590-6523-48f2-b66d-83e468dcd6e7" path="/var/lib/kubelet/pods/71504590-6523-48f2-b66d-83e468dcd6e7/volumes"
Sep 10 14:22:38.611669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.611640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzzvm\" (UniqueName: \"kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm\") pod \"336ccf94-7fee-4ac2-975a-486e4b633e6f\" (UID: \"336ccf94-7fee-4ac2-975a-486e4b633e6f\") "
Sep 10 14:22:38.613815 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.613785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm" (OuterVolumeSpecName: "kube-api-access-xzzvm") pod "336ccf94-7fee-4ac2-975a-486e4b633e6f" (UID: "336ccf94-7fee-4ac2-975a-486e4b633e6f"). InnerVolumeSpecName "kube-api-access-xzzvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:38.712977 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:38.712942    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzzvm\" (UniqueName: \"kubernetes.io/projected/336ccf94-7fee-4ac2-975a-486e4b633e6f-kube-api-access-xzzvm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:39.238033 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.238000    2581 generic.go:356] "Generic (PLEG): container finished" podID="336ccf94-7fee-4ac2-975a-486e4b633e6f" containerID="ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655" exitCode=0
Sep 10 14:22:39.238546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.238044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7896445869-kthzw"
Sep 10 14:22:39.238546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.238088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7896445869-kthzw" event={"ID":"336ccf94-7fee-4ac2-975a-486e4b633e6f","Type":"ContainerDied","Data":"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"}
Sep 10 14:22:39.238546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.238127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7896445869-kthzw" event={"ID":"336ccf94-7fee-4ac2-975a-486e4b633e6f","Type":"ContainerDied","Data":"9162f22d44a8cf215925a76ca3bde8ba08f19371b2ee2001c1433a46d0d1759a"}
Sep 10 14:22:39.238546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.238148    2581 scope.go:117] "RemoveContainer" containerID="ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"
Sep 10 14:22:39.246826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.246809    2581 scope.go:117] "RemoveContainer" containerID="ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"
Sep 10 14:22:39.247108 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:39.247088    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655\": container with ID starting with ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655 not found: ID does not exist" containerID="ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"
Sep 10 14:22:39.247166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.247117    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655"} err="failed to get container status \"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655\": rpc error: code = NotFound desc = could not find container \"ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655\": container with ID starting with ee0e40468e3ffa465ee42427b815a629c6374abcf77a6781628fabf45b654655 not found: ID does not exist"
Sep 10 14:22:39.257811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.257781    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:39.259588 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:39.259561    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7896445869-kthzw"]
Sep 10 14:22:40.530390 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.530355    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="336ccf94-7fee-4ac2-975a-486e4b633e6f" path="/var/lib/kubelet/pods/336ccf94-7fee-4ac2-975a-486e4b633e6f/volumes"
Sep 10 14:22:40.943680 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943605    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:40.943936 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943924    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="336ccf94-7fee-4ac2-975a-486e4b633e6f" containerName="authorino"
Sep 10 14:22:40.943979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943938    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="336ccf94-7fee-4ac2-975a-486e4b633e6f" containerName="authorino"
Sep 10 14:22:40.943979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943950    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" containerName="authorino"
Sep 10 14:22:40.943979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943955    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" containerName="authorino"
Sep 10 14:22:40.943979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943967    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71504590-6523-48f2-b66d-83e468dcd6e7" containerName="authorino"
Sep 10 14:22:40.943979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.943972    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71504590-6523-48f2-b66d-83e468dcd6e7" containerName="authorino"
Sep 10 14:22:40.944131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.944026    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="336ccf94-7fee-4ac2-975a-486e4b633e6f" containerName="authorino"
Sep 10 14:22:40.944131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.944035    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad364ae-4d6a-4553-b426-d4d38dbbccf5" containerName="authorino"
Sep 10 14:22:40.944131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.944040    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71504590-6523-48f2-b66d-83e468dcd6e7" containerName="authorino"
Sep 10 14:22:40.947038 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.947017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:40.949789 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.949769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:22:40.949928 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.949900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t8r42\""
Sep 10 14:22:40.950003 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.949974    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:22:40.950058 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.950024    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:22:40.960860 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:40.960838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:41.033380 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.033346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqzhs\" (UniqueName: \"kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.033549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.033405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.033549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.033440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.117917 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.117882    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:22:41.120134 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.120115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.134194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.134159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:22:41.134304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.134224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.134304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.134269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.134384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.134328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqzhs\" (UniqueName: \"kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.134384 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:41.134374    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 14:22:41.134451 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:41.134431    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert podName:475b9b02-58da-456b-a06b-a07da7a175f8 nodeName:}" failed. No retries permitted until 2026-09-10 14:22:41.634410752 +0000 UTC m=+673.595920255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert") pod "maas-controller-78fcdbf9ff-tdddw" (UID: "475b9b02-58da-456b-a06b-a07da7a175f8") : secret "maas-controller-webhook-cert" not found
Sep 10 14:22:41.136773 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.136746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.146769 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.146584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqzhs\" (UniqueName: \"kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.234819 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.234731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.234819 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.234774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76js\" (UniqueName: \"kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.234999 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.234890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.335879 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.335840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.335879 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.335888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76js\" (UniqueName: \"kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.336225 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.336022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.336225 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:41.336136    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 14:22:41.336225 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:41.336203    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert podName:170def59-59d9-4e5f-aac7-17ca255c293f nodeName:}" failed. No retries permitted until 2026-09-10 14:22:41.836182237 +0000 UTC m=+673.797691735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert") pod "maas-controller-764f45744-zsgfj" (UID: "170def59-59d9-4e5f-aac7-17ca255c293f") : secret "maas-controller-webhook-cert" not found
Sep 10 14:22:41.338497 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.338473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.346715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.346667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76js\" (UniqueName: \"kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.639384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.639351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.641845 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.641822    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") pod \"maas-controller-78fcdbf9ff-tdddw\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") " pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:41.840432 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.840398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.842741 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.842722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") pod \"maas-controller-764f45744-zsgfj\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") " pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:41.856673 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:41.856648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:42.030501 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:42.030464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:42.125413 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:42.125356    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:42.127430 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:42.127400    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475b9b02_58da_456b_a06b_a07da7a175f8.slice/crio-9ed4f88aeda3b6b252b3863cc7cc8f1c45f2b6500462dfa069116950ca28bf56 WatchSource:0}: Error finding container 9ed4f88aeda3b6b252b3863cc7cc8f1c45f2b6500462dfa069116950ca28bf56: Status 404 returned error can't find the container with id 9ed4f88aeda3b6b252b3863cc7cc8f1c45f2b6500462dfa069116950ca28bf56
Sep 10 14:22:42.252900 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:42.252815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" event={"ID":"475b9b02-58da-456b-a06b-a07da7a175f8","Type":"ContainerStarted","Data":"9ed4f88aeda3b6b252b3863cc7cc8f1c45f2b6500462dfa069116950ca28bf56"}
Sep 10 14:22:42.297409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:42.297387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:22:42.299690 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:22:42.299662    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170def59_59d9_4e5f_aac7_17ca255c293f.slice/crio-7b7a7bbe9199961e8937d1a0b3740cbce6b8566f6d9747997cce80c71815240a WatchSource:0}: Error finding container 7b7a7bbe9199961e8937d1a0b3740cbce6b8566f6d9747997cce80c71815240a: Status 404 returned error can't find the container with id 7b7a7bbe9199961e8937d1a0b3740cbce6b8566f6d9747997cce80c71815240a
Sep 10 14:22:43.267591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:43.267503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-zsgfj" event={"ID":"170def59-59d9-4e5f-aac7-17ca255c293f","Type":"ContainerStarted","Data":"7b7a7bbe9199961e8937d1a0b3740cbce6b8566f6d9747997cce80c71815240a"}
Sep 10 14:22:45.279416 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.279376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" event={"ID":"475b9b02-58da-456b-a06b-a07da7a175f8","Type":"ContainerStarted","Data":"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"}
Sep 10 14:22:45.279888 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.279498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:45.281056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.281033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-zsgfj" event={"ID":"170def59-59d9-4e5f-aac7-17ca255c293f","Type":"ContainerStarted","Data":"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"}
Sep 10 14:22:45.281151 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.281076    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:45.300204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.300148    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" podStartSLOduration=2.635651067 podStartE2EDuration="5.300131009s" totalImagesPullingTime="2.664479942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:45.298225789 +0000 UTC m=+677.259735308" watchObservedRunningTime="2026-09-10 14:22:45.300131009 +0000 UTC m=+677.261640525"
Sep 10 14:22:45.324987 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:45.324929    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-764f45744-zsgfj" podStartSLOduration=1.8238861979999998 podStartE2EDuration="4.324912434s" totalImagesPullingTime="2.501026236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:45.322218164 +0000 UTC m=+677.283727684" watchObservedRunningTime="2026-09-10 14:22:45.324912434 +0000 UTC m=+677.286421952"
Sep 10 14:22:56.290023 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.289992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:56.290494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.290063    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:22:56.349966 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.349932    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:56.350156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.350113    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" podUID="475b9b02-58da-456b-a06b-a07da7a175f8" containerName="manager" containerID="cri-o://22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94" gracePeriod=30
Sep 10 14:22:56.589325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.589301    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:56.674220 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.674191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs\") pod \"475b9b02-58da-456b-a06b-a07da7a175f8\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") "
Sep 10 14:22:56.674404 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.674296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") pod \"475b9b02-58da-456b-a06b-a07da7a175f8\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") "
Sep 10 14:22:56.674404 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.674349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqzhs\" (UniqueName: \"kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs\") pod \"475b9b02-58da-456b-a06b-a07da7a175f8\" (UID: \"475b9b02-58da-456b-a06b-a07da7a175f8\") "
Sep 10 14:22:56.676406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.676346    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "475b9b02-58da-456b-a06b-a07da7a175f8" (UID: "475b9b02-58da-456b-a06b-a07da7a175f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:22:56.676525 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.676426    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "475b9b02-58da-456b-a06b-a07da7a175f8" (UID: "475b9b02-58da-456b-a06b-a07da7a175f8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:22:56.676783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.676767    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs" (OuterVolumeSpecName: "kube-api-access-nqzhs") pod "475b9b02-58da-456b-a06b-a07da7a175f8" (UID: "475b9b02-58da-456b-a06b-a07da7a175f8"). InnerVolumeSpecName "kube-api-access-nqzhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:22:56.775475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.775437    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-webhook-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:56.775475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.775470    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqzhs\" (UniqueName: \"kubernetes.io/projected/475b9b02-58da-456b-a06b-a07da7a175f8-kube-api-access-nqzhs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:56.775475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:56.775479    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/475b9b02-58da-456b-a06b-a07da7a175f8-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:22:57.339745 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.339672    2581 generic.go:356] "Generic (PLEG): container finished" podID="475b9b02-58da-456b-a06b-a07da7a175f8" containerID="22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94" exitCode=0
Sep 10 14:22:57.339745 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.339733    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw"
Sep 10 14:22:57.340203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.339767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" event={"ID":"475b9b02-58da-456b-a06b-a07da7a175f8","Type":"ContainerDied","Data":"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"}
Sep 10 14:22:57.340203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.339817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78fcdbf9ff-tdddw" event={"ID":"475b9b02-58da-456b-a06b-a07da7a175f8","Type":"ContainerDied","Data":"9ed4f88aeda3b6b252b3863cc7cc8f1c45f2b6500462dfa069116950ca28bf56"}
Sep 10 14:22:57.340203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.339843    2581 scope.go:117] "RemoveContainer" containerID="22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"
Sep 10 14:22:57.348948 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.348930    2581 scope.go:117] "RemoveContainer" containerID="22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"
Sep 10 14:22:57.349226 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:22:57.349208    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94\": container with ID starting with 22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94 not found: ID does not exist" containerID="22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"
Sep 10 14:22:57.349280 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.349234    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94"} err="failed to get container status \"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94\": rpc error: code = NotFound desc = could not find container \"22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94\": container with ID starting with 22eb6e55990bd0d8e2895e48852e3cab63236637dcf72e32b67a40b434786c94 not found: ID does not exist"
Sep 10 14:22:57.360127 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.360095    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:57.364098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:57.364068    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78fcdbf9ff-tdddw"]
Sep 10 14:22:58.531551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:22:58.531515    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="475b9b02-58da-456b-a06b-a07da7a175f8" path="/var/lib/kubelet/pods/475b9b02-58da-456b-a06b-a07da7a175f8/volumes"
Sep 10 14:23:18.917798 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.917756    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:23:18.918292 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.918271    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475b9b02-58da-456b-a06b-a07da7a175f8" containerName="manager"
Sep 10 14:23:18.918336 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.918296    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="475b9b02-58da-456b-a06b-a07da7a175f8" containerName="manager"
Sep 10 14:23:18.918399 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.918389    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="475b9b02-58da-456b-a06b-a07da7a175f8" containerName="manager"
Sep 10 14:23:18.923440 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.923417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:18.925963 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.925938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 14:23:18.926100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.926041    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:23:18.931887 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.931861    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:23:18.963441 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.963410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:18.963621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.963460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:18.963621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.963546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:18.963621 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:18.963617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9xqs\" (UniqueName: \"kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.064919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.064877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.065106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.064934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9xqs\" (UniqueName: \"kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.065106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.064992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.065106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.065017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.065106 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:23:19.065039    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 14:23:19.065316 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:23:19.065121    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls podName:3b5d1d7a-1e3e-4324-a6f0-57747afcefa8 nodeName:}" failed. No retries permitted until 2026-09-10 14:23:19.565099477 +0000 UTC m=+711.526608981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls") pod "maas-api-7cd87849b-8nzwv" (UID: "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8") : secret "maas-api-serving-cert" not found
Sep 10 14:23:19.065635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.065614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.067553 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.067526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.073990 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.073946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9xqs\" (UniqueName: \"kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.407408 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.407359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"]
Sep 10 14:23:19.409743 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.409723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.412166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.412145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 14:23:19.421677 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.421647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"]
Sep 10 14:23:19.469548 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.469500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.469736 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.469717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4l2x\" (UniqueName: \"kubernetes.io/projected/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-kube-api-access-z4l2x\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.571319 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.571276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.571511 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.571401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.571511 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.571462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4l2x\" (UniqueName: \"kubernetes.io/projected/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-kube-api-access-z4l2x\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.572095 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.572069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.574072 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.574044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") pod \"maas-api-7cd87849b-8nzwv\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.580256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.580229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4l2x\" (UniqueName: \"kubernetes.io/projected/b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8-kube-api-access-z4l2x\") pod \"payload-pre-processing-59fbc47bf8-spm9g\" (UID: \"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.721168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.721079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:19.835348 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.835314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:19.998670 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:19.998589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"]
Sep 10 14:23:20.002567 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:23:20.002531    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f37565_3bd6_4ac5_bb56_f5d3a8f45ec8.slice/crio-6c33cacfb8abfdc02888d6cfdaaf3098b962d27122ee2f4e9ed3d94c1479e7af WatchSource:0}: Error finding container 6c33cacfb8abfdc02888d6cfdaaf3098b962d27122ee2f4e9ed3d94c1479e7af: Status 404 returned error can't find the container with id 6c33cacfb8abfdc02888d6cfdaaf3098b962d27122ee2f4e9ed3d94c1479e7af
Sep 10 14:23:20.110452 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:20.110422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:23:20.111539 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:23:20.111506    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5d1d7a_1e3e_4324_a6f0_57747afcefa8.slice/crio-ab8d60b322fbbc472be15c6503d138db5acaa271119ab7ce0803cb36311aa1b9 WatchSource:0}: Error finding container ab8d60b322fbbc472be15c6503d138db5acaa271119ab7ce0803cb36311aa1b9: Status 404 returned error can't find the container with id ab8d60b322fbbc472be15c6503d138db5acaa271119ab7ce0803cb36311aa1b9
Sep 10 14:23:20.449938 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:20.449900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g" event={"ID":"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8","Type":"ContainerStarted","Data":"6c33cacfb8abfdc02888d6cfdaaf3098b962d27122ee2f4e9ed3d94c1479e7af"}
Sep 10 14:23:20.451151 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:20.451125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" event={"ID":"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8","Type":"ContainerStarted","Data":"ab8d60b322fbbc472be15c6503d138db5acaa271119ab7ce0803cb36311aa1b9"}
Sep 10 14:23:23.468721 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.468660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g" event={"ID":"b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8","Type":"ContainerStarted","Data":"8b4a58e3727421c6caad2381a5bc50349a00ded54e66d47cf842ba07b1de9a7f"}
Sep 10 14:23:23.469197 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.468815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:23.470199 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.470168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" event={"ID":"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8","Type":"ContainerStarted","Data":"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"}
Sep 10 14:23:23.470327 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.470257    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:23.484579 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.484525    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g" podStartSLOduration=2.041679426 podStartE2EDuration="4.48450882s" totalImagesPullingTime="2.442829394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:23:23.483635647 +0000 UTC m=+715.445145148" watchObservedRunningTime="2026-09-10 14:23:23.48450882 +0000 UTC m=+715.446018369"
Sep 10 14:23:23.501106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:23.501044    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" podStartSLOduration=3.172300438 podStartE2EDuration="5.501022072s" totalImagesPullingTime="2.328721634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:23:23.498355544 +0000 UTC m=+715.459865062" watchObservedRunningTime="2026-09-10 14:23:23.501022072 +0000 UTC m=+715.462531593"
Sep 10 14:23:29.481198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:29.481164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:23:34.477147 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:34.477111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spm9g"
Sep 10 14:23:41.107603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.107562    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"]
Sep 10 14:23:41.112456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.112427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.115037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.115010    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 14:23:41.115273 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.115257    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 14:23:41.116614 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.116577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4c25c\""
Sep 10 14:23:41.116751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.116659    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 14:23:41.118663 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.118641    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"]
Sep 10 14:23:41.162775 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.162944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzcw\" (UniqueName: \"kubernetes.io/projected/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-kube-api-access-qtzcw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.162944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.162944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.162944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.163100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.162973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.263885 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.263850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzcw\" (UniqueName: \"kubernetes.io/projected/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-kube-api-access-qtzcw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.263891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.263921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.263951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.263992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.264026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264326 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.264314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264404 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.264380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.264453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.264415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.266336 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.266318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.266589 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.266572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.276458 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.276428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzcw\" (UniqueName: \"kubernetes.io/projected/ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9-kube-api-access-qtzcw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5mc7r\" (UID: \"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.424015 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.423925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:41.698344 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.698319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"]
Sep 10 14:23:41.700242 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:23:41.700211    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb37ea6_9ab3_4a5f_8599_0b5e827fb5f9.slice/crio-8082e2aad4c2a330c4014b1eececfb8612da93a6891fc3fb36cd272c209eb52e WatchSource:0}: Error finding container 8082e2aad4c2a330c4014b1eececfb8612da93a6891fc3fb36cd272c209eb52e: Status 404 returned error can't find the container with id 8082e2aad4c2a330c4014b1eececfb8612da93a6891fc3fb36cd272c209eb52e
Sep 10 14:23:41.701663 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:41.701642    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:23:42.560546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:42.560510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r" event={"ID":"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9","Type":"ContainerStarted","Data":"8082e2aad4c2a330c4014b1eececfb8612da93a6891fc3fb36cd272c209eb52e"}
Sep 10 14:23:43.566501 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:43.566465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r" event={"ID":"ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9","Type":"ContainerStarted","Data":"9ac16d3a62285c274cc1c8001866f737249849120e5fb3338dbb592ec12be30b"}
Sep 10 14:23:43.583593 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:43.583536    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r" podStartSLOduration=1.479165285 podStartE2EDuration="2.583520929s" totalImagesPullingTime="1.104355644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:23:43.581503358 +0000 UTC m=+735.543012890" watchObservedRunningTime="2026-09-10 14:23:43.583520929 +0000 UTC m=+735.545030444"
Sep 10 14:23:45.815207 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.815176    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"]
Sep 10 14:23:45.818608 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.818588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.821364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.821343    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 14:23:45.827917 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.827894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"]
Sep 10 14:23:45.906080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.906080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb00c14b-cc97-475b-afc9-399fbed512c2-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.906300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.906300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.906300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:45.906300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:45.906231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8zmj\" (UniqueName: \"kubernetes.io/projected/cb00c14b-cc97-475b-afc9-399fbed512c2-kube-api-access-g8zmj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007434 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007434 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb00c14b-cc97-475b-afc9-399fbed512c2-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8zmj\" (UniqueName: \"kubernetes.io/projected/cb00c14b-cc97-475b-afc9-399fbed512c2-kube-api-access-g8zmj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007933 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007997 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.007997 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.007982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.009815 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.009797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb00c14b-cc97-475b-afc9-399fbed512c2-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.010204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.010186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb00c14b-cc97-475b-afc9-399fbed512c2-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.016582 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.016557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8zmj\" (UniqueName: \"kubernetes.io/projected/cb00c14b-cc97-475b-afc9-399fbed512c2-kube-api-access-g8zmj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv\" (UID: \"cb00c14b-cc97-475b-afc9-399fbed512c2\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.129269 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.129180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:46.405093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.405068    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"]
Sep 10 14:23:46.407039 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:23:46.407010    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb00c14b_cc97_475b_afc9_399fbed512c2.slice/crio-625d03dc00319d0453ecccef10075c83aecbedbf21ae1a409769078a8fc39d52 WatchSource:0}: Error finding container 625d03dc00319d0453ecccef10075c83aecbedbf21ae1a409769078a8fc39d52: Status 404 returned error can't find the container with id 625d03dc00319d0453ecccef10075c83aecbedbf21ae1a409769078a8fc39d52
Sep 10 14:23:46.580887 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:46.580863    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv" event={"ID":"cb00c14b-cc97-475b-afc9-399fbed512c2","Type":"ContainerStarted","Data":"625d03dc00319d0453ecccef10075c83aecbedbf21ae1a409769078a8fc39d52"}
Sep 10 14:23:47.587160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:47.587123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv" event={"ID":"cb00c14b-cc97-475b-afc9-399fbed512c2","Type":"ContainerStarted","Data":"fe39b5165b5918f46500645e8b9f337b490aef3551412decfbd350946325d940"}
Sep 10 14:23:47.616822 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:47.616766    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv" podStartSLOduration=2.446239714 podStartE2EDuration="2.616748604s" totalImagesPullingTime="170.50889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:23:47.614140646 +0000 UTC m=+739.575650163" watchObservedRunningTime="2026-09-10 14:23:47.616748604 +0000 UTC m=+739.578258121"
Sep 10 14:23:51.424789 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:51.424747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:51.425204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:51.424812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:51.437475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:51.437435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:51.617203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:51.617174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5mc7r"
Sep 10 14:23:56.129577 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:56.129548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:56.129983 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:56.129879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:56.142123 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:56.142103    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:23:56.639571 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:23:56.639543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv"
Sep 10 14:24:34.702868 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.702831    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"]
Sep 10 14:24:34.705297 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.705274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.707852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.707826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 14:24:34.717046 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.717023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"]
Sep 10 14:24:34.809129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.809129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.809346 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s954m\" (UniqueName: \"kubernetes.io/projected/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-kube-api-access-s954m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.809346 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.809346 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.809453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.809343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910514 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910514 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s954m\" (UniqueName: \"kubernetes.io/projected/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-kube-api-access-s954m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910952 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.910952 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.910914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.911160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.911053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.911160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.911138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.912942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.912910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.913048 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.913021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:34.918406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:34.918376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s954m\" (UniqueName: \"kubernetes.io/projected/ba573bee-2de2-4e82-ad2f-60697ee7a8cb-kube-api-access-s954m\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-p6stq\" (UID: \"ba573bee-2de2-4e82-ad2f-60697ee7a8cb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:35.016440 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:35.016400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:35.285694 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:35.285666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"]
Sep 10 14:24:35.287553 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:24:35.287527    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba573bee_2de2_4e82_ad2f_60697ee7a8cb.slice/crio-6886c809f4c1a43636995549549c7864bf942c6804da8264e6fd8894d9254e7a WatchSource:0}: Error finding container 6886c809f4c1a43636995549549c7864bf942c6804da8264e6fd8894d9254e7a: Status 404 returned error can't find the container with id 6886c809f4c1a43636995549549c7864bf942c6804da8264e6fd8894d9254e7a
Sep 10 14:24:35.805743 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:35.805634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq" event={"ID":"ba573bee-2de2-4e82-ad2f-60697ee7a8cb","Type":"ContainerStarted","Data":"eea9cc3a5036db72a9c0ec7f3474573b74b5a650f8ee83c9a03878c7cfdb4a5f"}
Sep 10 14:24:35.805743 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:35.805693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq" event={"ID":"ba573bee-2de2-4e82-ad2f-60697ee7a8cb","Type":"ContainerStarted","Data":"6886c809f4c1a43636995549549c7864bf942c6804da8264e6fd8894d9254e7a"}
Sep 10 14:24:35.824798 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:35.824754    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq" podStartSLOduration=1.583760436 podStartE2EDuration="1.824740171s" totalImagesPullingTime="240.979735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:35.822873232 +0000 UTC m=+787.784382747" watchObservedRunningTime="2026-09-10 14:24:35.824740171 +0000 UTC m=+787.786249688"
Sep 10 14:24:45.016767 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:45.016724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:45.017183 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:45.016786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:45.029195 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:45.029169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:24:45.864171 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:24:45.864140    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-p6stq"
Sep 10 14:25:55.683290 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.683257    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-685577bfc4-mwhhg"]
Sep 10 14:25:55.685911 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.685888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.695411 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.695387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685577bfc4-mwhhg"]
Sep 10 14:25:55.719903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.719873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/804fa8e8-d78f-49ce-b312-4aefbafb8169-tls-cert\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.720054 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.719920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbmzq\" (UniqueName: \"kubernetes.io/projected/804fa8e8-d78f-49ce-b312-4aefbafb8169-kube-api-access-kbmzq\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.820461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.820421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/804fa8e8-d78f-49ce-b312-4aefbafb8169-tls-cert\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.820461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.820471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbmzq\" (UniqueName: \"kubernetes.io/projected/804fa8e8-d78f-49ce-b312-4aefbafb8169-kube-api-access-kbmzq\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.822952 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.822926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/804fa8e8-d78f-49ce-b312-4aefbafb8169-tls-cert\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.828676 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.828654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbmzq\" (UniqueName: \"kubernetes.io/projected/804fa8e8-d78f-49ce-b312-4aefbafb8169-kube-api-access-kbmzq\") pod \"authorino-685577bfc4-mwhhg\" (UID: \"804fa8e8-d78f-49ce-b312-4aefbafb8169\") " pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:55.995901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:55.995805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685577bfc4-mwhhg"
Sep 10 14:25:56.265530 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:56.265507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685577bfc4-mwhhg"]
Sep 10 14:25:56.267861 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:25:56.267836    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod804fa8e8_d78f_49ce_b312_4aefbafb8169.slice/crio-288ebb1f323749b84c1890b11dd065c448a99fa2e9dddc31c43a31e6ce6fa600 WatchSource:0}: Error finding container 288ebb1f323749b84c1890b11dd065c448a99fa2e9dddc31c43a31e6ce6fa600: Status 404 returned error can't find the container with id 288ebb1f323749b84c1890b11dd065c448a99fa2e9dddc31c43a31e6ce6fa600
Sep 10 14:25:57.180276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.180185    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685577bfc4-mwhhg" event={"ID":"804fa8e8-d78f-49ce-b312-4aefbafb8169","Type":"ContainerStarted","Data":"352233ad65e7146015354752350a9ba68c6f03dde55ce1cf90b6ac25ed2c8e9b"}
Sep 10 14:25:57.180276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.180245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685577bfc4-mwhhg" event={"ID":"804fa8e8-d78f-49ce-b312-4aefbafb8169","Type":"ContainerStarted","Data":"288ebb1f323749b84c1890b11dd065c448a99fa2e9dddc31c43a31e6ce6fa600"}
Sep 10 14:25:57.199635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.199575    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-685577bfc4-mwhhg" podStartSLOduration=1.582194672 podStartE2EDuration="2.199558882s" totalImagesPullingTime="617.36421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:57.197390664 +0000 UTC m=+869.158900184" watchObservedRunningTime="2026-09-10 14:25:57.199558882 +0000 UTC m=+869.161068399"
Sep 10 14:25:57.233861 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.233690    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:25:57.234273 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.234242    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b7879d44c-mpt2s" podUID="784c7c62-511c-4785-ab45-7c67f2ea5677" containerName="authorino" containerID="cri-o://0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84" gracePeriod=30
Sep 10 14:25:57.490070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.490039    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:25:57.540090 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.540061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert\") pod \"784c7c62-511c-4785-ab45-7c67f2ea5677\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") "
Sep 10 14:25:57.540277 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.540140    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzdtt\" (UniqueName: \"kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt\") pod \"784c7c62-511c-4785-ab45-7c67f2ea5677\" (UID: \"784c7c62-511c-4785-ab45-7c67f2ea5677\") "
Sep 10 14:25:57.542290 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.542257    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt" (OuterVolumeSpecName: "kube-api-access-hzdtt") pod "784c7c62-511c-4785-ab45-7c67f2ea5677" (UID: "784c7c62-511c-4785-ab45-7c67f2ea5677"). InnerVolumeSpecName "kube-api-access-hzdtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:25:57.550595 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.550567    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "784c7c62-511c-4785-ab45-7c67f2ea5677" (UID: "784c7c62-511c-4785-ab45-7c67f2ea5677"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:25:57.641970 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.641935    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/784c7c62-511c-4785-ab45-7c67f2ea5677-tls-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:25:57.641970 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:57.641964    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzdtt\" (UniqueName: \"kubernetes.io/projected/784c7c62-511c-4785-ab45-7c67f2ea5677-kube-api-access-hzdtt\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:25:58.187266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.187171    2581 generic.go:356] "Generic (PLEG): container finished" podID="784c7c62-511c-4785-ab45-7c67f2ea5677" containerID="0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84" exitCode=0
Sep 10 14:25:58.187266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.187206    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b7879d44c-mpt2s"
Sep 10 14:25:58.187266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.187252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b7879d44c-mpt2s" event={"ID":"784c7c62-511c-4785-ab45-7c67f2ea5677","Type":"ContainerDied","Data":"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"}
Sep 10 14:25:58.187865 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.187308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b7879d44c-mpt2s" event={"ID":"784c7c62-511c-4785-ab45-7c67f2ea5677","Type":"ContainerDied","Data":"18bde5750bb892f6d905e51db1f490baed8c153cc69e19da8b8e425f2c334c16"}
Sep 10 14:25:58.187865 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.187334    2581 scope.go:117] "RemoveContainer" containerID="0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"
Sep 10 14:25:58.199785 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.199721    2581 scope.go:117] "RemoveContainer" containerID="0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"
Sep 10 14:25:58.200270 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:25:58.200248    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84\": container with ID starting with 0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84 not found: ID does not exist" containerID="0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"
Sep 10 14:25:58.200372 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.200280    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84"} err="failed to get container status \"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84\": rpc error: code = NotFound desc = could not find container \"0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84\": container with ID starting with 0bd840f03a56c23157242552927b097504f62f97d1942b2fd7877949993cab84 not found: ID does not exist"
Sep 10 14:25:58.210650 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.210624    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:25:58.215971 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.215948    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b7879d44c-mpt2s"]
Sep 10 14:25:58.531520 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:25:58.531488    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784c7c62-511c-4785-ab45-7c67f2ea5677" path="/var/lib/kubelet/pods/784c7c62-511c-4785-ab45-7c67f2ea5677/volumes"
Sep 10 14:26:26.985338 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.985306    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:26:26.985870 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.985669    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784c7c62-511c-4785-ab45-7c67f2ea5677" containerName="authorino"
Sep 10 14:26:26.985870 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.985682    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="784c7c62-511c-4785-ab45-7c67f2ea5677" containerName="authorino"
Sep 10 14:26:26.985870 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.985770    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="784c7c62-511c-4785-ab45-7c67f2ea5677" containerName="authorino"
Sep 10 14:26:26.988792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.988774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:26.992079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.992054    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:26:26.992516 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:26.992498    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-bb42d5-serving-cert\""
Sep 10 14:26:27.000307 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.000242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:26:27.115586 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.115550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hjv\" (UniqueName: \"kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.115780 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.115613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.115780 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.115646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.115780 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.115694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.216195 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.216159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.216350 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:26:27.216315    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-serving-cert: secret "maas-api-e2e-inf-a-bb42d5-serving-cert" not found
Sep 10 14:26:27.216397 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:26:27.216383    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls podName:acf1871c-9060-4e07-93f4-8baa540dfc0b nodeName:}" failed. No retries permitted until 2026-09-10 14:26:27.716368349 +0000 UTC m=+899.677877849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls") pod "maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" (UID: "acf1871c-9060-4e07-93f4-8baa540dfc0b") : secret "maas-api-e2e-inf-a-bb42d5-serving-cert" not found
Sep 10 14:26:27.216442 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.216314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hjv\" (UniqueName: \"kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.216493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.216471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.216533 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.216493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.217045 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.217026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.218802 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.218782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.232364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.232343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hjv\" (UniqueName: \"kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.720772 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.720734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.723551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.723515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.807384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.807351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:26:27.811111 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.811091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:27.814338 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.814314    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bb42d5-dockercfg-fkqzb\""
Sep 10 14:26:27.815874 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.815856    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bb42d5\""
Sep 10 14:26:27.826162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.826138    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:26:27.901761 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.901727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:27.922298 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.922264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxkxk\" (UniqueName: \"kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:27.922442 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:27.922393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.023545 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.023515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxkxk\" (UniqueName: \"kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.023950 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.023577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.024241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.024219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.032942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.032920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxkxk\" (UniqueName: \"kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk\") pod \"payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.122851 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.122818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:28.203044 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.203021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:26:28.205756 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:28.205729    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf1871c_9060_4e07_93f4_8baa540dfc0b.slice/crio-a35d0471de04e7f346d5cc0bc3b85ad3d1d6cff5f7ed483675fa3fe45ae9465f WatchSource:0}: Error finding container a35d0471de04e7f346d5cc0bc3b85ad3d1d6cff5f7ed483675fa3fe45ae9465f: Status 404 returned error can't find the container with id a35d0471de04e7f346d5cc0bc3b85ad3d1d6cff5f7ed483675fa3fe45ae9465f
Sep 10 14:26:28.325345 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.325251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" event={"ID":"acf1871c-9060-4e07-93f4-8baa540dfc0b","Type":"ContainerStarted","Data":"a35d0471de04e7f346d5cc0bc3b85ad3d1d6cff5f7ed483675fa3fe45ae9465f"}
Sep 10 14:26:28.395603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.395551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:26:28.398463 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:28.398434    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddece904f_93a3_4cd3_9573_7cebaf280d6e.slice/crio-8a2ccfd05e5ce297d22ced73ec72053a41135c0793c7452127732bc0ddcdc289 WatchSource:0}: Error finding container 8a2ccfd05e5ce297d22ced73ec72053a41135c0793c7452127732bc0ddcdc289: Status 404 returned error can't find the container with id 8a2ccfd05e5ce297d22ced73ec72053a41135c0793c7452127732bc0ddcdc289
Sep 10 14:26:28.486438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.486403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:26:28.487956 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:28.487933    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:26:29.331744 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.331689    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" event={"ID":"dece904f-93a3-4cd3-9573-7cebaf280d6e","Type":"ContainerStarted","Data":"fb42fb3682dd73bfda3a4814d77babe7ed55e7709ba89d41561d10bfe725c896"}
Sep 10 14:26:29.332188 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.331770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:29.332188 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.331794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" event={"ID":"dece904f-93a3-4cd3-9573-7cebaf280d6e","Type":"ContainerStarted","Data":"8a2ccfd05e5ce297d22ced73ec72053a41135c0793c7452127732bc0ddcdc289"}
Sep 10 14:26:29.333303 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.333273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" event={"ID":"acf1871c-9060-4e07-93f4-8baa540dfc0b","Type":"ContainerStarted","Data":"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"}
Sep 10 14:26:29.333432 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.333326    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:29.348894 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.348848    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" podStartSLOduration=2.348834274 podStartE2EDuration="2.348834274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:29.346776092 +0000 UTC m=+901.308285611" watchObservedRunningTime="2026-09-10 14:26:29.348834274 +0000 UTC m=+901.310343791"
Sep 10 14:26:29.364488 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:29.364445    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" podStartSLOduration=2.6644509530000002 podStartE2EDuration="3.364429203s" totalImagesPullingTime="699.97825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:29.362426465 +0000 UTC m=+901.323935983" watchObservedRunningTime="2026-09-10 14:26:29.364429203 +0000 UTC m=+901.325938722"
Sep 10 14:26:34.184076 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.184040    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:26:34.188428 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.188403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.191036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.191011    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2bae5943-serving-cert\""
Sep 10 14:26:34.196921 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.196896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:26:34.294874 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.294839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mlvl\" (UniqueName: \"kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.295055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.294898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.295055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.294931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.295055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.294952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.395919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.395857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mlvl\" (UniqueName: \"kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.396110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.395962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.396110 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.396005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.396205 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.396131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.396803 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.396783    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.398565 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.398544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.398669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.398569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.404197 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.404177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mlvl\" (UniqueName: \"kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl\") pod \"maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.502976 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.502888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:34.778018 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:34.777995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:26:34.779731 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:34.779692    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4660d286_479a_48ac_9177_0e36f92f1a28.slice/crio-30546b895e2486d4c355458e64c49bbaa4c2ed42ff68bd5cb2149493fd9d194e WatchSource:0}: Error finding container 30546b895e2486d4c355458e64c49bbaa4c2ed42ff68bd5cb2149493fd9d194e: Status 404 returned error can't find the container with id 30546b895e2486d4c355458e64c49bbaa4c2ed42ff68bd5cb2149493fd9d194e
Sep 10 14:26:35.342611 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:35.342582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:26:35.361030 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:35.360990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" event={"ID":"4660d286-479a-48ac-9177-0e36f92f1a28","Type":"ContainerStarted","Data":"30546b895e2486d4c355458e64c49bbaa4c2ed42ff68bd5cb2149493fd9d194e"}
Sep 10 14:26:36.368519 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:36.368481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" event={"ID":"4660d286-479a-48ac-9177-0e36f92f1a28","Type":"ContainerStarted","Data":"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"}
Sep 10 14:26:36.368938 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:36.368551    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:36.387858 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:36.387808    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" podStartSLOduration=1.6677519950000002 podStartE2EDuration="2.387792252s" totalImagesPullingTime="720.040257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:36.387114153 +0000 UTC m=+908.348623670" watchObservedRunningTime="2026-09-10 14:26:36.387792252 +0000 UTC m=+908.349301769"
Sep 10 14:26:40.339962 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:40.339931    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:26:42.380280 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:42.380252    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:26:46.163064 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.163015    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:26:46.167021 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.166997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.169775 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.169753    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-44e3af-serving-cert\""
Sep 10 14:26:46.183898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.183872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:26:46.207665 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.207637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.207836 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.207677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.207836 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.207708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4tf2\" (UniqueName: \"kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.207836 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.207754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.309165 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.309130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.309373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.309175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.309373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.309196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4tf2\" (UniqueName: \"kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.309373 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.309216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.309373 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:26:46.309298    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-serving-cert: secret "maas-api-e2e-autores-44e3af-serving-cert" not found
Sep 10 14:26:46.309594 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:26:46.309382    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls podName:cce730a2-e348-4ec4-872c-76bd30b00817 nodeName:}" failed. No retries permitted until 2026-09-10 14:26:46.809361383 +0000 UTC m=+918.770870893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls") pod "maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" (UID: "cce730a2-e348-4ec4-872c-76bd30b00817") : secret "maas-api-e2e-autores-44e3af-serving-cert" not found
Sep 10 14:26:46.309934 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.309916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.311556 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.311540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.320688 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.320663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4tf2\" (UniqueName: \"kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.793937 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.793901    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:26:46.798361 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.798341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:46.801072 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.801042    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-44e3af\""
Sep 10 14:26:46.801177 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.801089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-44e3af-dockercfg-qp4vs\""
Sep 10 14:26:46.806568 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.806539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:26:46.813657 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.813635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.816189 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.816168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") pod \"maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:46.914408 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.914369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:46.914627 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:46.914431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnfh6\" (UniqueName: \"kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.015677 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.015636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnfh6\" (UniqueName: \"kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.015907 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.015792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.016498 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.016474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.024949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.024927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnfh6\" (UniqueName: \"kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6\") pod \"payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") " pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.077443 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.077362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:47.111585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.111552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:47.353389 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.353365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:26:47.356031 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:47.355994    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce730a2_e348_4ec4_872c_76bd30b00817.slice/crio-d201b12fe14e6bbe9fd64a7b179a47228b345aa14bb17be9a309bae80bed9f27 WatchSource:0}: Error finding container d201b12fe14e6bbe9fd64a7b179a47228b345aa14bb17be9a309bae80bed9f27: Status 404 returned error can't find the container with id d201b12fe14e6bbe9fd64a7b179a47228b345aa14bb17be9a309bae80bed9f27
Sep 10 14:26:47.395278 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.395159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:26:47.398241 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:47.398207    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6d97b2_3431_42c4_8195_0ccb142a1332.slice/crio-41bb586019159583391cf1fef58283a059f3b0ad648ab332ee0c225e302ceb51 WatchSource:0}: Error finding container 41bb586019159583391cf1fef58283a059f3b0ad648ab332ee0c225e302ceb51: Status 404 returned error can't find the container with id 41bb586019159583391cf1fef58283a059f3b0ad648ab332ee0c225e302ceb51
Sep 10 14:26:47.433109 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.433071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" event={"ID":"cce730a2-e348-4ec4-872c-76bd30b00817","Type":"ContainerStarted","Data":"d201b12fe14e6bbe9fd64a7b179a47228b345aa14bb17be9a309bae80bed9f27"}
Sep 10 14:26:47.434327 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:47.434297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" event={"ID":"5b6d97b2-3431-42c4-8195-0ccb142a1332","Type":"ContainerStarted","Data":"41bb586019159583391cf1fef58283a059f3b0ad648ab332ee0c225e302ceb51"}
Sep 10 14:26:48.442034 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.441990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" event={"ID":"cce730a2-e348-4ec4-872c-76bd30b00817","Type":"ContainerStarted","Data":"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"}
Sep 10 14:26:48.442535 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.442065    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:48.443580 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.443553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" event={"ID":"5b6d97b2-3431-42c4-8195-0ccb142a1332","Type":"ContainerStarted","Data":"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"}
Sep 10 14:26:48.443736 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.443685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:26:48.462545 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.462492    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" podStartSLOduration=2.021666794 podStartE2EDuration="2.462476745s" totalImagesPullingTime="440.809951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:48.461335514 +0000 UTC m=+920.422845045" watchObservedRunningTime="2026-09-10 14:26:48.462476745 +0000 UTC m=+920.423986305"
Sep 10 14:26:48.479351 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:48.479305    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" podStartSLOduration=2.479289736 podStartE2EDuration="2.479289736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:48.477611689 +0000 UTC m=+920.439121199" watchObservedRunningTime="2026-09-10 14:26:48.479289736 +0000 UTC m=+920.440799253"
Sep 10 14:26:54.453667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:54.453634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:26:56.855211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.855179    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:26:56.858604 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.858585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:56.861351 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.861332    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-8f0c5f\""
Sep 10 14:26:56.861351 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.861343    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-8f0c5f-dockercfg-dg92d\""
Sep 10 14:26:56.874032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.874005    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:26:56.892106 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.892077    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:26:56.897238 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.897214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:56.907224 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:56.907195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:26:57.012954 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.012910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwqn\" (UniqueName: \"kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.013184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.013091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.013184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.013144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.013302 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.013192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2h78\" (UniqueName: \"kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.113875 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.113784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.113875 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.113832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.114101 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.113928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2h78\" (UniqueName: \"kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.114101 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.113958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwqn\" (UniqueName: \"kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.114508 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.114486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.114611 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.114592    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.122719 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.122683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwqn\" (UniqueName: \"kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn\") pod \"payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.122882 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.122861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2h78\" (UniqueName: \"kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78\") pod \"payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.168624 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.168592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:57.212907 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.212871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:57.443211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.443186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:26:57.445809 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:57.445781    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9887ef_06b8_4310_a8b9_0afac29cafd9.slice/crio-c5fdc813c9da6592d27cf6b5cf239fc549d1ae31cb138b51abf6135973cf3fd1 WatchSource:0}: Error finding container c5fdc813c9da6592d27cf6b5cf239fc549d1ae31cb138b51abf6135973cf3fd1: Status 404 returned error can't find the container with id c5fdc813c9da6592d27cf6b5cf239fc549d1ae31cb138b51abf6135973cf3fd1
Sep 10 14:26:57.491971 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.491941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" event={"ID":"2a9887ef-06b8-4310-a8b9-0afac29cafd9","Type":"ContainerStarted","Data":"c5fdc813c9da6592d27cf6b5cf239fc549d1ae31cb138b51abf6135973cf3fd1"}
Sep 10 14:26:57.504029 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:57.504002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:26:57.505803 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:26:57.505769    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbed3d9_d05c_49fc_9893_3347e96aacd1.slice/crio-5af090c4938d1e216f55c39d765a54e225e38b0b73d439a263118ec85544d863 WatchSource:0}: Error finding container 5af090c4938d1e216f55c39d765a54e225e38b0b73d439a263118ec85544d863: Status 404 returned error can't find the container with id 5af090c4938d1e216f55c39d765a54e225e38b0b73d439a263118ec85544d863
Sep 10 14:26:58.498472 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.498279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" event={"ID":"2a9887ef-06b8-4310-a8b9-0afac29cafd9","Type":"ContainerStarted","Data":"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"}
Sep 10 14:26:58.498472 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.498349    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:26:58.500253 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.500224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" event={"ID":"3dbed3d9-d05c-49fc-9893-3347e96aacd1","Type":"ContainerStarted","Data":"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"}
Sep 10 14:26:58.500377 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.500282    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" event={"ID":"3dbed3d9-d05c-49fc-9893-3347e96aacd1","Type":"ContainerStarted","Data":"5af090c4938d1e216f55c39d765a54e225e38b0b73d439a263118ec85544d863"}
Sep 10 14:26:58.500377 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.500309    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:26:58.516831 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.516784    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" podStartSLOduration=2.516769962 podStartE2EDuration="2.516769962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:58.513796944 +0000 UTC m=+930.475306464" watchObservedRunningTime="2026-09-10 14:26:58.516769962 +0000 UTC m=+930.478279480"
Sep 10 14:26:58.544065 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:58.544009    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" podStartSLOduration=2.543991122 podStartE2EDuration="2.543991122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:58.542999256 +0000 UTC m=+930.504508776" watchObservedRunningTime="2026-09-10 14:26:58.543991122 +0000 UTC m=+930.505500640"
Sep 10 14:26:59.451148 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:26:59.451121    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:27:09.508920 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:09.508884    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:27:09.509310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:09.509056    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:27:14.871447 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:14.869119    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:27:14.871447 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:14.869449    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" podUID="4660d286-479a-48ac-9177-0e36f92f1a28" containerName="maas-api" containerID="cri-o://9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa" gracePeriod=30
Sep 10 14:27:15.133039 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.132972    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:27:15.283601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.283562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mlvl\" (UniqueName: \"kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl\") pod \"4660d286-479a-48ac-9177-0e36f92f1a28\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") "
Sep 10 14:27:15.283601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.283596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs\") pod \"4660d286-479a-48ac-9177-0e36f92f1a28\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") "
Sep 10 14:27:15.283882 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.283638    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls\") pod \"4660d286-479a-48ac-9177-0e36f92f1a28\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") "
Sep 10 14:27:15.283882 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.283755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca\") pod \"4660d286-479a-48ac-9177-0e36f92f1a28\" (UID: \"4660d286-479a-48ac-9177-0e36f92f1a28\") "
Sep 10 14:27:15.284244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.284213    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4660d286-479a-48ac-9177-0e36f92f1a28" (UID: "4660d286-479a-48ac-9177-0e36f92f1a28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:27:15.285908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.285883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4660d286-479a-48ac-9177-0e36f92f1a28" (UID: "4660d286-479a-48ac-9177-0e36f92f1a28"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:27:15.285975 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.285961    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4660d286-479a-48ac-9177-0e36f92f1a28" (UID: "4660d286-479a-48ac-9177-0e36f92f1a28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:27:15.286020 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.285960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl" (OuterVolumeSpecName: "kube-api-access-7mlvl") pod "4660d286-479a-48ac-9177-0e36f92f1a28" (UID: "4660d286-479a-48ac-9177-0e36f92f1a28"). InnerVolumeSpecName "kube-api-access-7mlvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:27:15.385078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.384974    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4660d286-479a-48ac-9177-0e36f92f1a28-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:15.385078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.385021    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mlvl\" (UniqueName: \"kubernetes.io/projected/4660d286-479a-48ac-9177-0e36f92f1a28-kube-api-access-7mlvl\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:15.385078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.385031    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:15.385078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.385040    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4660d286-479a-48ac-9177-0e36f92f1a28-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:15.586989 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.586955    2581 generic.go:356] "Generic (PLEG): container finished" podID="4660d286-479a-48ac-9177-0e36f92f1a28" containerID="9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa" exitCode=0
Sep 10 14:27:15.587164 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.587005    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"
Sep 10 14:27:15.587164 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.587021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" event={"ID":"4660d286-479a-48ac-9177-0e36f92f1a28","Type":"ContainerDied","Data":"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"}
Sep 10 14:27:15.587164 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.587063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6" event={"ID":"4660d286-479a-48ac-9177-0e36f92f1a28","Type":"ContainerDied","Data":"30546b895e2486d4c355458e64c49bbaa4c2ed42ff68bd5cb2149493fd9d194e"}
Sep 10 14:27:15.587164 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.587083    2581 scope.go:117] "RemoveContainer" containerID="9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"
Sep 10 14:27:15.596183 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.596162    2581 scope.go:117] "RemoveContainer" containerID="9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"
Sep 10 14:27:15.596460 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:27:15.596441    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa\": container with ID starting with 9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa not found: ID does not exist" containerID="9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"
Sep 10 14:27:15.596535 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.596471    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa"} err="failed to get container status \"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa\": rpc error: code = NotFound desc = could not find container \"9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa\": container with ID starting with 9a82ded38056a48613135ca236fcc51ee4d41959bb7adde1066925376d8546fa not found: ID does not exist"
Sep 10 14:27:15.607994 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.607970    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:27:15.613933 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:15.613912    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bae5943-5dc999c798-bl9g6"]
Sep 10 14:27:16.532219 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:16.532187    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4660d286-479a-48ac-9177-0e36f92f1a28" path="/var/lib/kubelet/pods/4660d286-479a-48ac-9177-0e36f92f1a28/volumes"
Sep 10 14:27:18.958783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.958747    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:18.959417 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.959396    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4660d286-479a-48ac-9177-0e36f92f1a28" containerName="maas-api"
Sep 10 14:27:18.959466 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.959421    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4660d286-479a-48ac-9177-0e36f92f1a28" containerName="maas-api"
Sep 10 14:27:18.959555 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.959542    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4660d286-479a-48ac-9177-0e36f92f1a28" containerName="maas-api"
Sep 10 14:27:18.964536 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.964512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:18.967026 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.967002    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-59ea8af5-serving-cert\""
Sep 10 14:27:18.974953 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:18.974930    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:19.019930 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.019896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct9nf\" (UniqueName: \"kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.020098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.019952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.020098 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.019976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.020183 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.020100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.121322 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.121286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.121523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.121368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct9nf\" (UniqueName: \"kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.121523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.121425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.121523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.121452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.122174 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.122141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.123991 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.123970    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.124118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.124004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.133090 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.133053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct9nf\" (UniqueName: \"kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf\") pod \"maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.278478 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.278424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:19.552300 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.552255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:19.554684 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:27:19.554654    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3252fddb_77f3_4c8e_be89_d0e125ded35f.slice/crio-93baa7939fbf9c69ba605760f6da56da6cd506428004a5ce64eb24c9cb16d263 WatchSource:0}: Error finding container 93baa7939fbf9c69ba605760f6da56da6cd506428004a5ce64eb24c9cb16d263: Status 404 returned error can't find the container with id 93baa7939fbf9c69ba605760f6da56da6cd506428004a5ce64eb24c9cb16d263
Sep 10 14:27:19.608581 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:19.608541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" event={"ID":"3252fddb-77f3-4c8e-be89-d0e125ded35f","Type":"ContainerStarted","Data":"93baa7939fbf9c69ba605760f6da56da6cd506428004a5ce64eb24c9cb16d263"}
Sep 10 14:27:20.614980 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:20.614943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" event={"ID":"3252fddb-77f3-4c8e-be89-d0e125ded35f","Type":"ContainerStarted","Data":"2daec7fbee8c270bf1e2de605cec47eca5d2ec728872ebae96a3f692aacb49cf"}
Sep 10 14:27:20.615420 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:20.615093    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:20.632736 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:20.632665    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" podStartSLOduration=2.200315693 podStartE2EDuration="2.632650412s" totalImagesPullingTime="432.334719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:27:20.631822731 +0000 UTC m=+952.593332251" watchObservedRunningTime="2026-09-10 14:27:20.632650412 +0000 UTC m=+952.594159983"
Sep 10 14:27:26.626436 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:26.626403    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:50.535052 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.535021    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:50.535646 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.535229    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" podUID="3252fddb-77f3-4c8e-be89-d0e125ded35f" containerName="maas-api" containerID="cri-o://2daec7fbee8c270bf1e2de605cec47eca5d2ec728872ebae96a3f692aacb49cf" gracePeriod=30
Sep 10 14:27:50.761262 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.761228    2581 generic.go:356] "Generic (PLEG): container finished" podID="3252fddb-77f3-4c8e-be89-d0e125ded35f" containerID="2daec7fbee8c270bf1e2de605cec47eca5d2ec728872ebae96a3f692aacb49cf" exitCode=0
Sep 10 14:27:50.761436 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.761294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" event={"ID":"3252fddb-77f3-4c8e-be89-d0e125ded35f","Type":"ContainerDied","Data":"2daec7fbee8c270bf1e2de605cec47eca5d2ec728872ebae96a3f692aacb49cf"}
Sep 10 14:27:50.792913 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.792850    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:50.926093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.926004    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct9nf\" (UniqueName: \"kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf\") pod \"3252fddb-77f3-4c8e-be89-d0e125ded35f\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") "
Sep 10 14:27:50.926093 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.926051    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs\") pod \"3252fddb-77f3-4c8e-be89-d0e125ded35f\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") "
Sep 10 14:27:50.926345 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.926121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls\") pod \"3252fddb-77f3-4c8e-be89-d0e125ded35f\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") "
Sep 10 14:27:50.926345 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.926156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca\") pod \"3252fddb-77f3-4c8e-be89-d0e125ded35f\" (UID: \"3252fddb-77f3-4c8e-be89-d0e125ded35f\") "
Sep 10 14:27:50.926555 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.926529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3252fddb-77f3-4c8e-be89-d0e125ded35f" (UID: "3252fddb-77f3-4c8e-be89-d0e125ded35f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:27:50.928359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.928328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf" (OuterVolumeSpecName: "kube-api-access-ct9nf") pod "3252fddb-77f3-4c8e-be89-d0e125ded35f" (UID: "3252fddb-77f3-4c8e-be89-d0e125ded35f"). InnerVolumeSpecName "kube-api-access-ct9nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:27:50.928359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.928341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3252fddb-77f3-4c8e-be89-d0e125ded35f" (UID: "3252fddb-77f3-4c8e-be89-d0e125ded35f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:27:50.928535 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:50.928385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3252fddb-77f3-4c8e-be89-d0e125ded35f" (UID: "3252fddb-77f3-4c8e-be89-d0e125ded35f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:27:51.027601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.027558    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct9nf\" (UniqueName: \"kubernetes.io/projected/3252fddb-77f3-4c8e-be89-d0e125ded35f-kube-api-access-ct9nf\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:51.027601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.027594    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:51.027601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.027605    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3252fddb-77f3-4c8e-be89-d0e125ded35f-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:51.027601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.027619    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3252fddb-77f3-4c8e-be89-d0e125ded35f-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:27:51.767821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.767790    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"
Sep 10 14:27:51.767821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.767815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8" event={"ID":"3252fddb-77f3-4c8e-be89-d0e125ded35f","Type":"ContainerDied","Data":"93baa7939fbf9c69ba605760f6da56da6cd506428004a5ce64eb24c9cb16d263"}
Sep 10 14:27:51.768319 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.767875    2581 scope.go:117] "RemoveContainer" containerID="2daec7fbee8c270bf1e2de605cec47eca5d2ec728872ebae96a3f692aacb49cf"
Sep 10 14:27:51.796512 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.796485    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:51.800961 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:51.800937    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59ea8af5-5db6fcc548-vjhf8"]
Sep 10 14:27:52.532049 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:27:52.532013    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3252fddb-77f3-4c8e-be89-d0e125ded35f" path="/var/lib/kubelet/pods/3252fddb-77f3-4c8e-be89-d0e125ded35f/volumes"
Sep 10 14:28:50.634734 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.634690    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:28:50.635125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.635111    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3252fddb-77f3-4c8e-be89-d0e125ded35f" containerName="maas-api"
Sep 10 14:28:50.635125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.635122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3252fddb-77f3-4c8e-be89-d0e125ded35f" containerName="maas-api"
Sep 10 14:28:50.635204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.635198    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3252fddb-77f3-4c8e-be89-d0e125ded35f" containerName="maas-api"
Sep 10 14:28:50.639464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.639447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.642239 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.642213    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-44e3af\"/\"openshift-service-ca.crt\""
Sep 10 14:28:50.643325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.643305    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-44e3af\"/\"default-dockercfg-tfjrp\""
Sep 10 14:28:50.643468 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.643378    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-44e3af\"/\"kube-root-ca.crt\""
Sep 10 14:28:50.643468 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.643409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-44e3af\"/\"autores-model-44e3af-kserve-self-signed-certs\""
Sep 10 14:28:50.648041 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.647621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:28:50.672164 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.672310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7q5\" (UniqueName: \"kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.672310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.672310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.672422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.672487 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.672474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7q5\" (UniqueName: \"kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.773783 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.773773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.775785 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.775766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.776045 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.776029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.782461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.782439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7q5\" (UniqueName: \"kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5\") pod \"autores-model-44e3af-kserve-55d4f66cdc-wcc47\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") " pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:50.951128 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:50.951030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:28:51.223546 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:51.223521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:28:51.225445 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:28:51.225400    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c8ad189_5417_4cc6_9ca5_6a657a3958cd.slice/crio-e0acfc0470f17d6edc8b75d124f861a9af82ae771273b50e88e5ef362139818e WatchSource:0}: Error finding container e0acfc0470f17d6edc8b75d124f861a9af82ae771273b50e88e5ef362139818e: Status 404 returned error can't find the container with id e0acfc0470f17d6edc8b75d124f861a9af82ae771273b50e88e5ef362139818e
Sep 10 14:28:52.059522 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:52.059489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" event={"ID":"3c8ad189-5417-4cc6-9ca5-6a657a3958cd","Type":"ContainerStarted","Data":"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"}
Sep 10 14:28:52.059969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:52.059531    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" event={"ID":"3c8ad189-5417-4cc6-9ca5-6a657a3958cd","Type":"ContainerStarted","Data":"e0acfc0470f17d6edc8b75d124f861a9af82ae771273b50e88e5ef362139818e"}
Sep 10 14:28:52.077895 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:52.077843    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" podStartSLOduration=2.077827894 podStartE2EDuration="2.077827894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:52.075508657 +0000 UTC m=+1044.037018174" watchObservedRunningTime="2026-09-10 14:28:52.077827894 +0000 UTC m=+1044.039337412"
Sep 10 14:28:57.906839 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.906801    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:28:57.910612 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.910588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.914841 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.914819    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bb42d5\"/\"kube-root-ca.crt\""
Sep 10 14:28:57.914841 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.914847    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bb42d5\"/\"openshift-service-ca.crt\""
Sep 10 14:28:57.915042 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.914847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bb42d5\"/\"test-model-bb42d5-kserve-self-signed-certs\""
Sep 10 14:28:57.915042 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.914900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bb42d5\"/\"default-dockercfg-txldq\""
Sep 10 14:28:57.920180 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.920158    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:28:57.945433 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.945614 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.945614 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.945614 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnvsf\" (UniqueName: \"kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.945614 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:57.945880 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:57.945626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.046558 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.046778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.046778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.046778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.046778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.047010 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnvsf\" (UniqueName: \"kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.047010 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.046973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.047116 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.047036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.047116 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.047099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.048906 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.048886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.049168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.049151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.054666 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.054641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnvsf\" (UniqueName: \"kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf\") pod \"test-model-bb42d5-kserve-bcb676cd7-cll4x\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") " pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.220944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.220860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:28:58.494033 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:58.494004    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:28:58.495501 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:28:58.495477    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0462d05_8840_4c49_b09e_6138b751b0fb.slice/crio-13168c5602effb2210d2fa21e60462ee68ddcd1104f5fa2bec46e480ea14e878 WatchSource:0}: Error finding container 13168c5602effb2210d2fa21e60462ee68ddcd1104f5fa2bec46e480ea14e878: Status 404 returned error can't find the container with id 13168c5602effb2210d2fa21e60462ee68ddcd1104f5fa2bec46e480ea14e878
Sep 10 14:28:59.095642 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:59.095591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" event={"ID":"f0462d05-8840-4c49-b09e-6138b751b0fb","Type":"ContainerStarted","Data":"de35afb89f70a247604faa9835ff4ff1b9f9e544b29bbbf33c9b0bf3c04105cb"}
Sep 10 14:28:59.096052 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:59.095652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" event={"ID":"f0462d05-8840-4c49-b09e-6138b751b0fb","Type":"ContainerStarted","Data":"13168c5602effb2210d2fa21e60462ee68ddcd1104f5fa2bec46e480ea14e878"}
Sep 10 14:28:59.114310 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:28:59.114260    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" podStartSLOduration=2.114244828 podStartE2EDuration="2.114244828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:59.112178836 +0000 UTC m=+1051.073688343" watchObservedRunningTime="2026-09-10 14:28:59.114244828 +0000 UTC m=+1051.075754346"
Sep 10 14:29:00.951173 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:00.951118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:00.951173 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:00.951183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:00.963735 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:00.963708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:01.117316 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:01.117287    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:04.122265 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:04.122229    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:29:08.221470 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:08.221430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:29:08.221874 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:08.221492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:29:08.234102 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:08.234080    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:29:09.161268 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:09.161235    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:29:14.706520 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:14.706484    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:29:14.707026 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:14.706781    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" podUID="cce730a2-e348-4ec4-872c-76bd30b00817" containerName="maas-api" containerID="cri-o://0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e" gracePeriod=30
Sep 10 14:29:14.763644 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:14.763600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:29:14.763881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:14.763851    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" podUID="5b6d97b2-3431-42c4-8195-0ccb142a1332" containerName="payload-processing" containerID="cri-o://18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538" gracePeriod=30
Sep 10 14:29:15.016545 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.016520    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:29:15.045398 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.045373    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:29:15.120478 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.120447    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") pod \"cce730a2-e348-4ec4-872c-76bd30b00817\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") "
Sep 10 14:29:15.120478 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.120478    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs\") pod \"cce730a2-e348-4ec4-872c-76bd30b00817\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") "
Sep 10 14:29:15.120677 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.120624    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4tf2\" (UniqueName: \"kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2\") pod \"cce730a2-e348-4ec4-872c-76bd30b00817\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") "
Sep 10 14:29:15.120744 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.120730    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca\") pod \"cce730a2-e348-4ec4-872c-76bd30b00817\" (UID: \"cce730a2-e348-4ec4-872c-76bd30b00817\") "
Sep 10 14:29:15.121070 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.121044    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cce730a2-e348-4ec4-872c-76bd30b00817" (UID: "cce730a2-e348-4ec4-872c-76bd30b00817"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:15.122571 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.122545    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cce730a2-e348-4ec4-872c-76bd30b00817" (UID: "cce730a2-e348-4ec4-872c-76bd30b00817"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:15.122661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.122621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cce730a2-e348-4ec4-872c-76bd30b00817" (UID: "cce730a2-e348-4ec4-872c-76bd30b00817"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:15.122661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.122623    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2" (OuterVolumeSpecName: "kube-api-access-z4tf2") pod "cce730a2-e348-4ec4-872c-76bd30b00817" (UID: "cce730a2-e348-4ec4-872c-76bd30b00817"). InnerVolumeSpecName "kube-api-access-z4tf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:15.181969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.181939    2581 generic.go:356] "Generic (PLEG): container finished" podID="cce730a2-e348-4ec4-872c-76bd30b00817" containerID="0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e" exitCode=0
Sep 10 14:29:15.182139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.181997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"
Sep 10 14:29:15.182139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.182024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" event={"ID":"cce730a2-e348-4ec4-872c-76bd30b00817","Type":"ContainerDied","Data":"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"}
Sep 10 14:29:15.182139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.182089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28" event={"ID":"cce730a2-e348-4ec4-872c-76bd30b00817","Type":"ContainerDied","Data":"d201b12fe14e6bbe9fd64a7b179a47228b345aa14bb17be9a309bae80bed9f27"}
Sep 10 14:29:15.182139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.182123    2581 scope.go:117] "RemoveContainer" containerID="0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"
Sep 10 14:29:15.183339 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.183316    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b6d97b2-3431-42c4-8195-0ccb142a1332" containerID="18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538" exitCode=0
Sep 10 14:29:15.183406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.183355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" event={"ID":"5b6d97b2-3431-42c4-8195-0ccb142a1332","Type":"ContainerDied","Data":"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"}
Sep 10 14:29:15.183406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.183395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck" event={"ID":"5b6d97b2-3431-42c4-8195-0ccb142a1332","Type":"ContainerDied","Data":"41bb586019159583391cf1fef58283a059f3b0ad648ab332ee0c225e302ceb51"}
Sep 10 14:29:15.183486 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.183360    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"
Sep 10 14:29:15.191757 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.191742    2581 scope.go:117] "RemoveContainer" containerID="0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"
Sep 10 14:29:15.192012 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:29:15.191992    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e\": container with ID starting with 0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e not found: ID does not exist" containerID="0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"
Sep 10 14:29:15.192085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.192024    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e"} err="failed to get container status \"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e\": rpc error: code = NotFound desc = could not find container \"0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e\": container with ID starting with 0fee92b11376094c22d112a0a30502ed6cc656c2027ac8e66ed346463586a07e not found: ID does not exist"
Sep 10 14:29:15.192085 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.192048    2581 scope.go:117] "RemoveContainer" containerID="18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"
Sep 10 14:29:15.199534 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.199515    2581 scope.go:117] "RemoveContainer" containerID="18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"
Sep 10 14:29:15.199879 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:29:15.199862    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538\": container with ID starting with 18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538 not found: ID does not exist" containerID="18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"
Sep 10 14:29:15.199945 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.199884    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538"} err="failed to get container status \"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538\": rpc error: code = NotFound desc = could not find container \"18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538\": container with ID starting with 18c068450c2647bf0dbe3c6ce7c426f51fafbbf19b2da5b4477cedbc005fd538 not found: ID does not exist"
Sep 10 14:29:15.204258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.204237    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:29:15.207517 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.207498    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-44e3af-5f6f4bb85-mjp28"]
Sep 10 14:29:15.221289 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume\") pod \"5b6d97b2-3431-42c4-8195-0ccb142a1332\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") "
Sep 10 14:29:15.221366 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnfh6\" (UniqueName: \"kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6\") pod \"5b6d97b2-3431-42c4-8195-0ccb142a1332\" (UID: \"5b6d97b2-3431-42c4-8195-0ccb142a1332\") "
Sep 10 14:29:15.221535 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221523    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.221573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221537    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cce730a2-e348-4ec4-872c-76bd30b00817-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.221573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221546    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4tf2\" (UniqueName: \"kubernetes.io/projected/cce730a2-e348-4ec4-872c-76bd30b00817-kube-api-access-z4tf2\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.221573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221555    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cce730a2-e348-4ec4-872c-76bd30b00817-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.221672 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.221627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b6d97b2-3431-42c4-8195-0ccb142a1332" (UID: "5b6d97b2-3431-42c4-8195-0ccb142a1332"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:15.223431 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.223414    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6" (OuterVolumeSpecName: "kube-api-access-nnfh6") pod "5b6d97b2-3431-42c4-8195-0ccb142a1332" (UID: "5b6d97b2-3431-42c4-8195-0ccb142a1332"). InnerVolumeSpecName "kube-api-access-nnfh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:15.322659 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.322624    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnfh6\" (UniqueName: \"kubernetes.io/projected/5b6d97b2-3431-42c4-8195-0ccb142a1332-kube-api-access-nnfh6\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.322659 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.322652    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b6d97b2-3431-42c4-8195-0ccb142a1332-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.504443 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.504405    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:29:15.509984 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:15.509948    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44e3af-5bc9b67ccf-gsgck"]
Sep 10 14:29:16.531628 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:16.531589    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6d97b2-3431-42c4-8195-0ccb142a1332" path="/var/lib/kubelet/pods/5b6d97b2-3431-42c4-8195-0ccb142a1332/volumes"
Sep 10 14:29:16.532077 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:16.532061    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cce730a2-e348-4ec4-872c-76bd30b00817" path="/var/lib/kubelet/pods/cce730a2-e348-4ec4-872c-76bd30b00817/volumes"
Sep 10 14:29:19.143304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.143263    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" podUID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" containerName="main" containerID="cri-o://7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94" gracePeriod=45
Sep 10 14:29:19.383753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.383728    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:19.564367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564329    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564560 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564560 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564453    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564560 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564560 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564805 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564614    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj7q5\" (UniqueName: \"kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5\") pod \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\" (UID: \"3c8ad189-5417-4cc6-9ca5-6a657a3958cd\") "
Sep 10 14:29:19.564805 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564727    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:19.564891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564803    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home" (OuterVolumeSpecName: "home") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:19.564891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564823    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:19.565007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564932    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-home\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:19.565007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564951    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-model-cache\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:19.565007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.564962    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tmp-dir\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:19.566644 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.566616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm" (OuterVolumeSpecName: "dshm") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:19.566644 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.566628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5" (OuterVolumeSpecName: "kube-api-access-sj7q5") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "kube-api-access-sj7q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:19.566644 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.566642    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c8ad189-5417-4cc6-9ca5-6a657a3958cd" (UID: "3c8ad189-5417-4cc6-9ca5-6a657a3958cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:19.666047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.666016    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj7q5\" (UniqueName: \"kubernetes.io/projected/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-kube-api-access-sj7q5\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:19.666047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.666043    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-tls-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:19.666047 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:19.666055    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8ad189-5417-4cc6-9ca5-6a657a3958cd-dshm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:29:20.210945 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.210908    2581 generic.go:356] "Generic (PLEG): container finished" podID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" containerID="7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94" exitCode=0
Sep 10 14:29:20.211364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.210964    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"
Sep 10 14:29:20.211364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.210992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" event={"ID":"3c8ad189-5417-4cc6-9ca5-6a657a3958cd","Type":"ContainerDied","Data":"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"}
Sep 10 14:29:20.211364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.211038    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47" event={"ID":"3c8ad189-5417-4cc6-9ca5-6a657a3958cd","Type":"ContainerDied","Data":"e0acfc0470f17d6edc8b75d124f861a9af82ae771273b50e88e5ef362139818e"}
Sep 10 14:29:20.211364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.211059    2581 scope.go:117] "RemoveContainer" containerID="7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"
Sep 10 14:29:20.219884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.219867    2581 scope.go:117] "RemoveContainer" containerID="7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"
Sep 10 14:29:20.220146 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:29:20.220126    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94\": container with ID starting with 7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94 not found: ID does not exist" containerID="7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"
Sep 10 14:29:20.220201 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.220154    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94"} err="failed to get container status \"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94\": rpc error: code = NotFound desc = could not find container \"7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94\": container with ID starting with 7b43b7f0f079fe4a7f3a86c290636a8f63ef420491c7bde87b1d901a14e32b94 not found: ID does not exist"
Sep 10 14:29:20.235268 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.235244    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:29:20.239656 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.239636    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-44e3af/autores-model-44e3af-kserve-55d4f66cdc-wcc47"]
Sep 10 14:29:20.531568 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:29:20.531534    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" path="/var/lib/kubelet/pods/3c8ad189-5417-4cc6-9ca5-6a657a3958cd/volumes"
Sep 10 14:30:00.143175 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143143    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"]
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce730a2-e348-4ec4-872c-76bd30b00817" containerName="maas-api"
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143563    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce730a2-e348-4ec4-872c-76bd30b00817" containerName="maas-api"
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143575    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b6d97b2-3431-42c4-8195-0ccb142a1332" containerName="payload-processing"
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143581    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b6d97b2-3431-42c4-8195-0ccb142a1332" containerName="payload-processing"
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143599    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" containerName="main"
Sep 10 14:30:00.143667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143605    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" containerName="main"
Sep 10 14:30:00.143901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143672    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c8ad189-5417-4cc6-9ca5-6a657a3958cd" containerName="main"
Sep 10 14:30:00.143901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143681    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce730a2-e348-4ec4-872c-76bd30b00817" containerName="maas-api"
Sep 10 14:30:00.143901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.143689    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b6d97b2-3431-42c4-8195-0ccb142a1332" containerName="payload-processing"
Sep 10 14:30:00.146695 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.146681    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"
Sep 10 14:30:00.149350 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.149329    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pj254\""
Sep 10 14:30:00.154464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.154443    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"]
Sep 10 14:30:00.157664 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.157645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"
Sep 10 14:30:00.428287 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.428266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"]
Sep 10 14:30:00.430372 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:30:00.430339    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c621520_44ce_476a_bba5_9282a5b0f486.slice/crio-d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b WatchSource:0}: Error finding container d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b: Status 404 returned error can't find the container with id d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b
Sep 10 14:30:00.432147 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:00.432129    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:30:01.415888 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:01.415853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs" event={"ID":"7c621520-44ce-476a-bba5-9282a5b0f486","Type":"ContainerStarted","Data":"d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b"}
Sep 10 14:30:04.432260 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.432161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs" event={"ID":"7c621520-44ce-476a-bba5-9282a5b0f486","Type":"ContainerStarted","Data":"7f03dacf57df2af037fba95b7c24c206c0b32ad1d5f71168106df6feee6ef8ce"}
Sep 10 14:30:04.447181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.447135    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs" podStartSLOduration=1.116963235 podStartE2EDuration="4.447121205s" totalImagesPullingTime="3.33015797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:04.446280295 +0000 UTC m=+1116.407789814" watchObservedRunningTime="2026-09-10 14:30:04.447121205 +0000 UTC m=+1116.408630723"
Sep 10 14:30:04.802264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.802226    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:04.805840 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.805818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.808617 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.808598    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ba183c\"/\"kube-root-ca.crt\""
Sep 10 14:30:04.808751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.808601    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ba183c\"/\"openshift-service-ca.crt\""
Sep 10 14:30:04.809867 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.809843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ba183c\"/\"explicit-model-ba183c-kserve-self-signed-certs\""
Sep 10 14:30:04.809980 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.809843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ba183c\"/\"default-dockercfg-pw9xc\""
Sep 10 14:30:04.815631 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.815343    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:04.975001 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.974965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.975174 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.975040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.975174 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.975069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.975174 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.975097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clzq5\" (UniqueName: \"kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.975174 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.975148    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:04.975316 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:04.975234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.075954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.075990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clzq5\" (UniqueName: \"kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076036 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076501 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.076555 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.076533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.078380 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.078351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.078658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.078572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.088396 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.088370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clzq5\" (UniqueName: \"kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5\") pod \"explicit-model-ba183c-kserve-bc8fdf95c-mrzpz\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") " pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.118769 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.118732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:05.388076 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.388040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:05.390167 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:30:05.390142    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb6dacdc_2fb1_4de9_a24b_e0c4cb88b964.slice/crio-c7dc9561bf033f3eba46b913f0951a4573f33b4f8bb89d173cf09d0e2f610a53 WatchSource:0}: Error finding container c7dc9561bf033f3eba46b913f0951a4573f33b4f8bb89d173cf09d0e2f610a53: Status 404 returned error can't find the container with id c7dc9561bf033f3eba46b913f0951a4573f33b4f8bb89d173cf09d0e2f610a53
Sep 10 14:30:05.438758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.438732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" event={"ID":"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964","Type":"ContainerStarted","Data":"c7dc9561bf033f3eba46b913f0951a4573f33b4f8bb89d173cf09d0e2f610a53"}
Sep 10 14:30:05.440309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.440288    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c621520-44ce-476a-bba5-9282a5b0f486" containerID="7f03dacf57df2af037fba95b7c24c206c0b32ad1d5f71168106df6feee6ef8ce" exitCode=0
Sep 10 14:30:05.440414 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:05.440372    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs" event={"ID":"7c621520-44ce-476a-bba5-9282a5b0f486","Type":"ContainerDied","Data":"7f03dacf57df2af037fba95b7c24c206c0b32ad1d5f71168106df6feee6ef8ce"}
Sep 10 14:30:06.448049 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:06.448011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" event={"ID":"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964","Type":"ContainerStarted","Data":"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"}
Sep 10 14:30:06.587171 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:06.587143    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"
Sep 10 14:30:06.602882 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:06.602809    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" podStartSLOduration=2.602788577 podStartE2EDuration="2.602788577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:06.46455787 +0000 UTC m=+1118.426067391" watchObservedRunningTime="2026-09-10 14:30:06.602788577 +0000 UTC m=+1118.564298097"
Sep 10 14:30:07.125497 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:07.125466    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:07.454156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:07.454080    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs"
Sep 10 14:30:07.454156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:07.454129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817510-dfnfs" event={"ID":"7c621520-44ce-476a-bba5-9282a5b0f486","Type":"ContainerDied","Data":"d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b"}
Sep 10 14:30:07.454569 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:07.454172    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1be50f209fbd2666beea869827aab034b68c2e42f2f7f95a2cdf8892fe7c78b"
Sep 10 14:30:07.600669 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:07.600643    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:07.600835 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:07.600734    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:08.100690445 +0000 UTC m=+1120.062199956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:08.107123 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:08.107079    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:08.107321 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:08.107182    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:09.10716025 +0000 UTC m=+1121.068669746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:09.115599 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:09.115564    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:09.116031 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:09.115640    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:11.115626359 +0000 UTC m=+1123.077135855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:11.133133 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:11.133097    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:11.133569 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:11.133172    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:15.133157183 +0000 UTC m=+1127.094666683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:15.119548 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:15.119515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:15.172825 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:15.172800    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:15.172970 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:15.172874    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:23.172859899 +0000 UTC m=+1135.134369395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:23.247770 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:23.247733    2581 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-self-signed-certs: secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:23.248301 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:23.247823    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs podName:bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964 nodeName:}" failed. No retries permitted until 2026-09-10 14:30:39.247804215 +0000 UTC m=+1151.209313717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs") pod "explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964") : secret "explicit-model-ba183c-kserve-self-signed-certs" not found
Sep 10 14:30:23.478801 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.478757    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" podUID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" containerName="main" containerID="cri-o://467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c" gracePeriod=45
Sep 10 14:30:23.732138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.732117    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:23.853138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clzq5\" (UniqueName: \"kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853305    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache\") pod \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\" (UID: \"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964\") "
Sep 10 14:30:23.853524 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853383    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home" (OuterVolumeSpecName: "home") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:23.853619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853581    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:23.853718 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853642    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:23.853718 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.853659    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-home\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:23.856044 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.856019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:30:23.856044 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.856026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm" (OuterVolumeSpecName: "dshm") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:23.856189 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.856119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5" (OuterVolumeSpecName: "kube-api-access-clzq5") pod "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" (UID: "bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964"). InnerVolumeSpecName "kube-api-access-clzq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:30:23.954898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.954853    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tmp-dir\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:23.954898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.954894    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-model-cache\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:23.954898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.954905    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clzq5\" (UniqueName: \"kubernetes.io/projected/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-kube-api-access-clzq5\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:23.954898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.954913    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-tls-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:23.955137 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:23.954922    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964-dshm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:30:24.536879 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.536844    2581 generic.go:356] "Generic (PLEG): container finished" podID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" containerID="467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c" exitCode=0
Sep 10 14:30:24.537309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.536882    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"
Sep 10 14:30:24.537309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.536902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" event={"ID":"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964","Type":"ContainerDied","Data":"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"}
Sep 10 14:30:24.537309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.536935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz" event={"ID":"bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964","Type":"ContainerDied","Data":"c7dc9561bf033f3eba46b913f0951a4573f33b4f8bb89d173cf09d0e2f610a53"}
Sep 10 14:30:24.537309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.536957    2581 scope.go:117] "RemoveContainer" containerID="467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"
Sep 10 14:30:24.550192 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.550173    2581 scope.go:117] "RemoveContainer" containerID="467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"
Sep 10 14:30:24.550445 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:30:24.550428    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c\": container with ID starting with 467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c not found: ID does not exist" containerID="467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"
Sep 10 14:30:24.550505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.550452    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c"} err="failed to get container status \"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c\": rpc error: code = NotFound desc = could not find container \"467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c\": container with ID starting with 467118a8d66d4f233fa8adfaff6ebc4a538d6b9f0efae47732cdab32bdc6bc6c not found: ID does not exist"
Sep 10 14:30:24.557058 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.557037    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:24.560491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:24.560461    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-ba183c/explicit-model-ba183c-kserve-bc8fdf95c-mrzpz"]
Sep 10 14:30:26.533166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:26.533132    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" path="/var/lib/kubelet/pods/bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964/volumes"
Sep 10 14:30:42.097629 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.097596    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.097991    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" containerName="main"
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.098002    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" containerName="main"
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.098023    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c621520-44ce-476a-bba5-9282a5b0f486" containerName="cleanup"
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.098028    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c621520-44ce-476a-bba5-9282a5b0f486" containerName="cleanup"
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.098090    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c621520-44ce-476a-bba5-9282a5b0f486" containerName="cleanup"
Sep 10 14:30:42.098168 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.098099    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb6dacdc-2fb1-4de9-a24b-e0c4cb88b964" containerName="main"
Sep 10 14:30:42.102579 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.102560    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.105567 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.105545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7c910967\""
Sep 10 14:30:42.105567 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.105561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7c910967-dockercfg-95jr8\""
Sep 10 14:30:42.111453 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.111431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:30:42.224896 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.224855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.224896 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.224898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrtqc\" (UniqueName: \"kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.326240 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.326196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrtqc\" (UniqueName: \"kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.326420 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.326317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.326999 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.326979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.335484 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.335456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrtqc\" (UniqueName: \"kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc\") pod \"payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") " pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.413897 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.413812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:42.686524 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:42.686481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:30:42.688675 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:30:42.688644    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5750239e_b32f_4e89_aac5_1cff7c39a8ef.slice/crio-4321717383a8bc0192eca92521218b2cc6e2c70a0d7975ce93aafd1f8a24e8c4 WatchSource:0}: Error finding container 4321717383a8bc0192eca92521218b2cc6e2c70a0d7975ce93aafd1f8a24e8c4: Status 404 returned error can't find the container with id 4321717383a8bc0192eca92521218b2cc6e2c70a0d7975ce93aafd1f8a24e8c4
Sep 10 14:30:43.627802 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:43.627765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" event={"ID":"5750239e-b32f-4e89-aac5-1cff7c39a8ef","Type":"ContainerStarted","Data":"9a51f4351bd7d714a6bf1b3e18ad0c6f41447047f041a271f0a58c9a76d90543"}
Sep 10 14:30:43.628216 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:43.627860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" event={"ID":"5750239e-b32f-4e89-aac5-1cff7c39a8ef","Type":"ContainerStarted","Data":"4321717383a8bc0192eca92521218b2cc6e2c70a0d7975ce93aafd1f8a24e8c4"}
Sep 10 14:30:43.628216 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:43.628016    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:30:43.647401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:43.647357    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" podStartSLOduration=1.6473409270000001 podStartE2EDuration="1.647340927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:43.646916637 +0000 UTC m=+1155.608426155" watchObservedRunningTime="2026-09-10 14:30:43.647340927 +0000 UTC m=+1155.608850449"
Sep 10 14:30:54.634424 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:30:54.634351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:31:01.615101 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:01.615061    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:31:13.493040 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.493000    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:31:13.493639 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.493256    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" podUID="acf1871c-9060-4e07-93f4-8baa540dfc0b" containerName="maas-api" containerID="cri-o://4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017" gracePeriod=30
Sep 10 14:31:13.623274 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.623238    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:31:13.623531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.623480    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" podUID="dece904f-93a3-4cd3-9573-7cebaf280d6e" containerName="payload-pre-processing" containerID="cri-o://fb42fb3682dd73bfda3a4814d77babe7ed55e7709ba89d41561d10bfe725c896" gracePeriod=30
Sep 10 14:31:13.778908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.778873    2581 generic.go:356] "Generic (PLEG): container finished" podID="dece904f-93a3-4cd3-9573-7cebaf280d6e" containerID="fb42fb3682dd73bfda3a4814d77babe7ed55e7709ba89d41561d10bfe725c896" exitCode=0
Sep 10 14:31:13.779038 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.778939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" event={"ID":"dece904f-93a3-4cd3-9573-7cebaf280d6e","Type":"ContainerDied","Data":"fb42fb3682dd73bfda3a4814d77babe7ed55e7709ba89d41561d10bfe725c896"}
Sep 10 14:31:13.876601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.876575    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:31:13.941386 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:13.941363    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:31:14.014585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume\") pod \"dece904f-93a3-4cd3-9573-7cebaf280d6e\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") "
Sep 10 14:31:14.014792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014592    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca\") pod \"acf1871c-9060-4e07-93f4-8baa540dfc0b\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") "
Sep 10 14:31:14.014792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") pod \"acf1871c-9060-4e07-93f4-8baa540dfc0b\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") "
Sep 10 14:31:14.014792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014691    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82hjv\" (UniqueName: \"kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv\") pod \"acf1871c-9060-4e07-93f4-8baa540dfc0b\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") "
Sep 10 14:31:14.014792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxkxk\" (UniqueName: \"kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk\") pod \"dece904f-93a3-4cd3-9573-7cebaf280d6e\" (UID: \"dece904f-93a3-4cd3-9573-7cebaf280d6e\") "
Sep 10 14:31:14.014792 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014785    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs\") pod \"acf1871c-9060-4e07-93f4-8baa540dfc0b\" (UID: \"acf1871c-9060-4e07-93f4-8baa540dfc0b\") "
Sep 10 14:31:14.015084 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.014984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "acf1871c-9060-4e07-93f4-8baa540dfc0b" (UID: "acf1871c-9060-4e07-93f4-8baa540dfc0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:14.015084 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.015013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dece904f-93a3-4cd3-9573-7cebaf280d6e" (UID: "dece904f-93a3-4cd3-9573-7cebaf280d6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:14.015203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.015127    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dece904f-93a3-4cd3-9573-7cebaf280d6e-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.015203 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.015143    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf1871c-9060-4e07-93f4-8baa540dfc0b-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.016893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.016860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv" (OuterVolumeSpecName: "kube-api-access-82hjv") pod "acf1871c-9060-4e07-93f4-8baa540dfc0b" (UID: "acf1871c-9060-4e07-93f4-8baa540dfc0b"). InnerVolumeSpecName "kube-api-access-82hjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:14.016893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.016868    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk" (OuterVolumeSpecName: "kube-api-access-lxkxk") pod "dece904f-93a3-4cd3-9573-7cebaf280d6e" (UID: "dece904f-93a3-4cd3-9573-7cebaf280d6e"). InnerVolumeSpecName "kube-api-access-lxkxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:14.017031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.016888    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "acf1871c-9060-4e07-93f4-8baa540dfc0b" (UID: "acf1871c-9060-4e07-93f4-8baa540dfc0b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:14.017031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.016867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "acf1871c-9060-4e07-93f4-8baa540dfc0b" (UID: "acf1871c-9060-4e07-93f4-8baa540dfc0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:14.116493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.116433    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxkxk\" (UniqueName: \"kubernetes.io/projected/dece904f-93a3-4cd3-9573-7cebaf280d6e-kube-api-access-lxkxk\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.116493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.116458    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.116493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.116468    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf1871c-9060-4e07-93f4-8baa540dfc0b-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.116493 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.116476    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82hjv\" (UniqueName: \"kubernetes.io/projected/acf1871c-9060-4e07-93f4-8baa540dfc0b-kube-api-access-82hjv\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:14.785202 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.785172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"
Sep 10 14:31:14.785655 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.785206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl" event={"ID":"dece904f-93a3-4cd3-9573-7cebaf280d6e","Type":"ContainerDied","Data":"8a2ccfd05e5ce297d22ced73ec72053a41135c0793c7452127732bc0ddcdc289"}
Sep 10 14:31:14.785655 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.785267    2581 scope.go:117] "RemoveContainer" containerID="fb42fb3682dd73bfda3a4814d77babe7ed55e7709ba89d41561d10bfe725c896"
Sep 10 14:31:14.786627 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.786540    2581 generic.go:356] "Generic (PLEG): container finished" podID="acf1871c-9060-4e07-93f4-8baa540dfc0b" containerID="4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017" exitCode=0
Sep 10 14:31:14.786627 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.786590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"
Sep 10 14:31:14.786753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.786640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" event={"ID":"acf1871c-9060-4e07-93f4-8baa540dfc0b","Type":"ContainerDied","Data":"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"}
Sep 10 14:31:14.786753 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.786675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk" event={"ID":"acf1871c-9060-4e07-93f4-8baa540dfc0b","Type":"ContainerDied","Data":"a35d0471de04e7f346d5cc0bc3b85ad3d1d6cff5f7ed483675fa3fe45ae9465f"}
Sep 10 14:31:14.800490 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.800472    2581 scope.go:117] "RemoveContainer" containerID="4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"
Sep 10 14:31:14.809768 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.809746    2581 scope.go:117] "RemoveContainer" containerID="4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"
Sep 10 14:31:14.810050 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:31:14.810029    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017\": container with ID starting with 4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017 not found: ID does not exist" containerID="4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"
Sep 10 14:31:14.810113 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.810059    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017"} err="failed to get container status \"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017\": rpc error: code = NotFound desc = could not find container \"4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017\": container with ID starting with 4cff6db267e1b9ccb878f93ce7c96a9efeac31ecc8de74dd1826df9daad76017 not found: ID does not exist"
Sep 10 14:31:14.811242 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.811199    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:31:14.813825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.813789    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb42d5-579764bdfd-tfmjl"]
Sep 10 14:31:14.823220 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.823202    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:31:14.826850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:14.826832    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb42d5-6894b6f58f-2thmk"]
Sep 10 14:31:16.532424 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.532388    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf1871c-9060-4e07-93f4-8baa540dfc0b" path="/var/lib/kubelet/pods/acf1871c-9060-4e07-93f4-8baa540dfc0b/volumes"
Sep 10 14:31:16.532810 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.532762    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dece904f-93a3-4cd3-9573-7cebaf280d6e" path="/var/lib/kubelet/pods/dece904f-93a3-4cd3-9573-7cebaf280d6e/volumes"
Sep 10 14:31:16.636760 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.636726    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" podUID="f0462d05-8840-4c49-b09e-6138b751b0fb" containerName="main" containerID="cri-o://de35afb89f70a247604faa9835ff4ff1b9f9e544b29bbbf33c9b0bf3c04105cb" gracePeriod=45
Sep 10 14:31:16.800129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.800044    2581 generic.go:356] "Generic (PLEG): container finished" podID="f0462d05-8840-4c49-b09e-6138b751b0fb" containerID="de35afb89f70a247604faa9835ff4ff1b9f9e544b29bbbf33c9b0bf3c04105cb" exitCode=0
Sep 10 14:31:16.800267 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.800119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" event={"ID":"f0462d05-8840-4c49-b09e-6138b751b0fb","Type":"ContainerDied","Data":"de35afb89f70a247604faa9835ff4ff1b9f9e544b29bbbf33c9b0bf3c04105cb"}
Sep 10 14:31:16.882041 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:16.882017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:31:17.040633 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.040633 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.040903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.040903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.040903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.040903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040812    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnvsf\" (UniqueName: \"kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf\") pod \"f0462d05-8840-4c49-b09e-6138b751b0fb\" (UID: \"f0462d05-8840-4c49-b09e-6138b751b0fb\") "
Sep 10 14:31:17.041091 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:17.041091 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.040955    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:17.041091 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.041006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home" (OuterVolumeSpecName: "home") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:17.041196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.041138    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-tmp-dir\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.041196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.041158    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-home\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.041196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.041173    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-model-cache\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.042919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.042898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm" (OuterVolumeSpecName: "dshm") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:17.043527 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.042941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:17.043527 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.043008    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf" (OuterVolumeSpecName: "kube-api-access-jnvsf") pod "f0462d05-8840-4c49-b09e-6138b751b0fb" (UID: "f0462d05-8840-4c49-b09e-6138b751b0fb"). InnerVolumeSpecName "kube-api-access-jnvsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:17.142376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.142298    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0462d05-8840-4c49-b09e-6138b751b0fb-dshm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.142376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.142324    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnvsf\" (UniqueName: \"kubernetes.io/projected/f0462d05-8840-4c49-b09e-6138b751b0fb-kube-api-access-jnvsf\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.142376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.142334    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0462d05-8840-4c49-b09e-6138b751b0fb-tls-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:17.806828 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.806796    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"
Sep 10 14:31:17.807279 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.806802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x" event={"ID":"f0462d05-8840-4c49-b09e-6138b751b0fb","Type":"ContainerDied","Data":"13168c5602effb2210d2fa21e60462ee68ddcd1104f5fa2bec46e480ea14e878"}
Sep 10 14:31:17.807279 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.806922    2581 scope.go:117] "RemoveContainer" containerID="de35afb89f70a247604faa9835ff4ff1b9f9e544b29bbbf33c9b0bf3c04105cb"
Sep 10 14:31:17.828352 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.828326    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:31:17.835496 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:17.835475    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-bb42d5/test-model-bb42d5-kserve-bcb676cd7-cll4x"]
Sep 10 14:31:18.531854 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:18.531812    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0462d05-8840-4c49-b09e-6138b751b0fb" path="/var/lib/kubelet/pods/f0462d05-8840-4c49-b09e-6138b751b0fb/volumes"
Sep 10 14:31:23.653278 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:23.653240    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:31:23.653874 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:23.653501    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" podUID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" containerName="payload-processing" containerID="cri-o://9a51f4351bd7d714a6bf1b3e18ad0c6f41447047f041a271f0a58c9a76d90543" gracePeriod=30
Sep 10 14:31:23.838829 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:23.838673    2581 generic.go:356] "Generic (PLEG): container finished" podID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" containerID="9a51f4351bd7d714a6bf1b3e18ad0c6f41447047f041a271f0a58c9a76d90543" exitCode=0
Sep 10 14:31:23.838829 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:23.838788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" event={"ID":"5750239e-b32f-4e89-aac5-1cff7c39a8ef","Type":"ContainerDied","Data":"9a51f4351bd7d714a6bf1b3e18ad0c6f41447047f041a271f0a58c9a76d90543"}
Sep 10 14:31:23.910537 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:23.910472    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:31:24.004927 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.004892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume\") pod \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") "
Sep 10 14:31:24.005114 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.004978    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrtqc\" (UniqueName: \"kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc\") pod \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\" (UID: \"5750239e-b32f-4e89-aac5-1cff7c39a8ef\") "
Sep 10 14:31:24.005263 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.005239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5750239e-b32f-4e89-aac5-1cff7c39a8ef" (UID: "5750239e-b32f-4e89-aac5-1cff7c39a8ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:24.006910 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.006889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc" (OuterVolumeSpecName: "kube-api-access-nrtqc") pod "5750239e-b32f-4e89-aac5-1cff7c39a8ef" (UID: "5750239e-b32f-4e89-aac5-1cff7c39a8ef"). InnerVolumeSpecName "kube-api-access-nrtqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:24.106365 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.106331    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrtqc\" (UniqueName: \"kubernetes.io/projected/5750239e-b32f-4e89-aac5-1cff7c39a8ef-kube-api-access-nrtqc\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:24.106365 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.106359    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5750239e-b32f-4e89-aac5-1cff7c39a8ef-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:24.845600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.845506    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"
Sep 10 14:31:24.845600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.845520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5" event={"ID":"5750239e-b32f-4e89-aac5-1cff7c39a8ef","Type":"ContainerDied","Data":"4321717383a8bc0192eca92521218b2cc6e2c70a0d7975ce93aafd1f8a24e8c4"}
Sep 10 14:31:24.845600 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.845591    2581 scope.go:117] "RemoveContainer" containerID="9a51f4351bd7d714a6bf1b3e18ad0c6f41447047f041a271f0a58c9a76d90543"
Sep 10 14:31:24.869095 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.869062    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:31:24.872514 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:24.872485    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7c910967-7d6b944d65-k4fq5"]
Sep 10 14:31:26.532053 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:26.532019    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" path="/var/lib/kubelet/pods/5750239e-b32f-4e89-aac5-1cff7c39a8ef/volumes"
Sep 10 14:31:28.517438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:28.517410    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:31:28.520692 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:28.520669    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:31:31.880923 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:31.880887    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:31:31.881422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:31.881099    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" podUID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" containerName="payload-processing" containerID="cri-o://c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641" gracePeriod=30
Sep 10 14:31:31.892531 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:31.892501    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:31:31.892769 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:31.892741    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" podUID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" containerName="payload-pre-processing" containerID="cri-o://b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9" gracePeriod=30
Sep 10 14:31:32.156402 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.156375    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:31:32.159849 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.159827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:31:32.282062 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282034    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume\") pod \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") "
Sep 10 14:31:32.282233 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282087    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqwqn\" (UniqueName: \"kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn\") pod \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") "
Sep 10 14:31:32.282233 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282110    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume\") pod \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\" (UID: \"3dbed3d9-d05c-49fc-9893-3347e96aacd1\") "
Sep 10 14:31:32.282233 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282170    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2h78\" (UniqueName: \"kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78\") pod \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\" (UID: \"2a9887ef-06b8-4310-a8b9-0afac29cafd9\") "
Sep 10 14:31:32.282549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282520    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3dbed3d9-d05c-49fc-9893-3347e96aacd1" (UID: "3dbed3d9-d05c-49fc-9893-3347e96aacd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:32.282549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.282535    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a9887ef-06b8-4310-a8b9-0afac29cafd9" (UID: "2a9887ef-06b8-4310-a8b9-0afac29cafd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:32.284223 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.284202    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn" (OuterVolumeSpecName: "kube-api-access-qqwqn") pod "3dbed3d9-d05c-49fc-9893-3347e96aacd1" (UID: "3dbed3d9-d05c-49fc-9893-3347e96aacd1"). InnerVolumeSpecName "kube-api-access-qqwqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:32.284331 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.284224    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78" (OuterVolumeSpecName: "kube-api-access-f2h78") pod "2a9887ef-06b8-4310-a8b9-0afac29cafd9" (UID: "2a9887ef-06b8-4310-a8b9-0afac29cafd9"). InnerVolumeSpecName "kube-api-access-f2h78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:32.383482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.383446    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9887ef-06b8-4310-a8b9-0afac29cafd9-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.383482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.383474    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqwqn\" (UniqueName: \"kubernetes.io/projected/3dbed3d9-d05c-49fc-9893-3347e96aacd1-kube-api-access-qqwqn\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.383482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.383483    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dbed3d9-d05c-49fc-9893-3347e96aacd1-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.383482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.383491    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2h78\" (UniqueName: \"kubernetes.io/projected/2a9887ef-06b8-4310-a8b9-0afac29cafd9-kube-api-access-f2h78\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.892563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.892479    2581 generic.go:356] "Generic (PLEG): container finished" podID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" containerID="c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641" exitCode=0
Sep 10 14:31:32.892563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.892540    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"
Sep 10 14:31:32.893081 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.892567    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" event={"ID":"3dbed3d9-d05c-49fc-9893-3347e96aacd1","Type":"ContainerDied","Data":"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"}
Sep 10 14:31:32.893081 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.892625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f" event={"ID":"3dbed3d9-d05c-49fc-9893-3347e96aacd1","Type":"ContainerDied","Data":"5af090c4938d1e216f55c39d765a54e225e38b0b73d439a263118ec85544d863"}
Sep 10 14:31:32.893081 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.892648    2581 scope.go:117] "RemoveContainer" containerID="c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"
Sep 10 14:31:32.894118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.893971    2581 generic.go:356] "Generic (PLEG): container finished" podID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" containerID="b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9" exitCode=0
Sep 10 14:31:32.894118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.894007    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"
Sep 10 14:31:32.894118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.894029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" event={"ID":"2a9887ef-06b8-4310-a8b9-0afac29cafd9","Type":"ContainerDied","Data":"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"}
Sep 10 14:31:32.894118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.894061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg" event={"ID":"2a9887ef-06b8-4310-a8b9-0afac29cafd9","Type":"ContainerDied","Data":"c5fdc813c9da6592d27cf6b5cf239fc549d1ae31cb138b51abf6135973cf3fd1"}
Sep 10 14:31:32.903927 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.903909    2581 scope.go:117] "RemoveContainer" containerID="c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"
Sep 10 14:31:32.904196 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:31:32.904178    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641\": container with ID starting with c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641 not found: ID does not exist" containerID="c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"
Sep 10 14:31:32.904244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.904204    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641"} err="failed to get container status \"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641\": rpc error: code = NotFound desc = could not find container \"c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641\": container with ID starting with c8d1402d5bb584e6211e2e18d08f2586dff0a5cf77967a6fc4f8b04107fc8641 not found: ID does not exist"
Sep 10 14:31:32.904244 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.904221    2581 scope.go:117] "RemoveContainer" containerID="b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"
Sep 10 14:31:32.913607 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.913472    2581 scope.go:117] "RemoveContainer" containerID="b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"
Sep 10 14:31:32.914107 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:31:32.914052    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9\": container with ID starting with b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9 not found: ID does not exist" containerID="b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"
Sep 10 14:31:32.914107 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.914087    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9"} err="failed to get container status \"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9\": rpc error: code = NotFound desc = could not find container \"b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9\": container with ID starting with b953095e49a7ec03dcd36b319424fb44d6942f1f94232521aaaffe781d29acf9 not found: ID does not exist"
Sep 10 14:31:32.916681 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.916658    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:31:32.920488 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.920467    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-8f0c5f-649c5566c4-w7ksg"]
Sep 10 14:31:32.930830 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.930809    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:31:32.940716 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:32.940675    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-8f0c5f-5945cd66b7-zg69f"]
Sep 10 14:31:34.530983 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:34.530939    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" path="/var/lib/kubelet/pods/2a9887ef-06b8-4310-a8b9-0afac29cafd9/volumes"
Sep 10 14:31:34.531476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:34.531437    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" path="/var/lib/kubelet/pods/3dbed3d9-d05c-49fc-9893-3347e96aacd1/volumes"
Sep 10 14:31:44.843645 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.843605    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:31:44.844260 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844239    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf1871c-9060-4e07-93f4-8baa540dfc0b" containerName="maas-api"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844264    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf1871c-9060-4e07-93f4-8baa540dfc0b" containerName="maas-api"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844278    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dece904f-93a3-4cd3-9573-7cebaf280d6e" containerName="payload-pre-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844288    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dece904f-93a3-4cd3-9573-7cebaf280d6e" containerName="payload-pre-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844305    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" containerName="payload-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844314    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" containerName="payload-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844327    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" containerName="payload-pre-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844335    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" containerName="payload-pre-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844350    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" containerName="payload-processing"
Sep 10 14:31:44.844354 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844358    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" containerName="payload-processing"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844370    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0462d05-8840-4c49-b09e-6138b751b0fb" containerName="main"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844378    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0462d05-8840-4c49-b09e-6138b751b0fb" containerName="main"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844452    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0462d05-8840-4c49-b09e-6138b751b0fb" containerName="main"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844471    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9887ef-06b8-4310-a8b9-0afac29cafd9" containerName="payload-pre-processing"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844479    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dbed3d9-d05c-49fc-9893-3347e96aacd1" containerName="payload-processing"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844485    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf1871c-9060-4e07-93f4-8baa540dfc0b" containerName="maas-api"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844491    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dece904f-93a3-4cd3-9573-7cebaf280d6e" containerName="payload-pre-processing"
Sep 10 14:31:44.844737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.844497    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5750239e-b32f-4e89-aac5-1cff7c39a8ef" containerName="payload-processing"
Sep 10 14:31:44.847326 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.847306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.850389 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.850366    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-586826-serving-cert\""
Sep 10 14:31:44.850495 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.850402    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:31:44.858598 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.858571    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:31:44.893547 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.893514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.893547 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.893550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.893761 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.893652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.893761 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.893695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg2f9\" (UniqueName: \"kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.994745 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.994682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.994922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.994759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg2f9\" (UniqueName: \"kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.994922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.994836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.994922 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.994861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.995109 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:31:44.995000    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-serving-cert: secret "maas-api-e2e-shared-a-w3-586826-serving-cert" not found
Sep 10 14:31:44.995109 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:31:44.995092    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls podName:1e3c24de-f263-4f2d-a265-cb2ea4a61dff nodeName:}" failed. No retries permitted until 2026-09-10 14:31:45.495058548 +0000 UTC m=+1217.456568043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls") pod "maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" (UID: "1e3c24de-f263-4f2d-a265-cb2ea4a61dff") : secret "maas-api-e2e-shared-a-w3-586826-serving-cert" not found
Sep 10 14:31:44.995474 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.995450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:44.997062 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:44.997041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:45.004674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.004651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg2f9\" (UniqueName: \"kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:45.480004 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.479971    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:31:45.482837 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.482816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.485482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.485461    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-586826\""
Sep 10 14:31:45.485482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.485473    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-586826-dockercfg-2mq9c\""
Sep 10 14:31:45.492653 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.492630    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:31:45.499312 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.499287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.499409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.499331    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24ksp\" (UniqueName: \"kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.499491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.499474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:45.501730 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.501692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:45.600347 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.600309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.600347 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.600353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24ksp\" (UniqueName: \"kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.601028 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.601007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.608578 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.608545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24ksp\" (UniqueName: \"kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp\") pod \"payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:45.760673 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.760577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:45.794707 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:45.794666    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:46.031223 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.031196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:31:46.033918 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:31:46.033890    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3c24de_f263_4f2d_a265_cb2ea4a61dff.slice/crio-feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab WatchSource:0}: Error finding container feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab: Status 404 returned error can't find the container with id feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab
Sep 10 14:31:46.072118 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.072094    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:31:46.075176 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:31:46.075148    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ba97a0_e0e7_47e5_831b_2b6c04ab530a.slice/crio-873443c0978eeac2b310559d866baccb98a2251ffffc3108ef4c2f07560bb046 WatchSource:0}: Error finding container 873443c0978eeac2b310559d866baccb98a2251ffffc3108ef4c2f07560bb046: Status 404 returned error can't find the container with id 873443c0978eeac2b310559d866baccb98a2251ffffc3108ef4c2f07560bb046
Sep 10 14:31:46.961582 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.961544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" event={"ID":"09ba97a0-e0e7-47e5-831b-2b6c04ab530a","Type":"ContainerStarted","Data":"de9db34f6ada58e9dd174f3110132d531f673d31cde4323d21a3b81325c96149"}
Sep 10 14:31:46.961843 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.961727    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:31:46.961843 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.961764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" event={"ID":"09ba97a0-e0e7-47e5-831b-2b6c04ab530a","Type":"ContainerStarted","Data":"873443c0978eeac2b310559d866baccb98a2251ffffc3108ef4c2f07560bb046"}
Sep 10 14:31:46.963198 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.963174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" event={"ID":"1e3c24de-f263-4f2d-a265-cb2ea4a61dff","Type":"ContainerStarted","Data":"455f065fea3a7a339668bd12a61156aa6feacdf857023651b586304a19421d4a"}
Sep 10 14:31:46.963295 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.963211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" event={"ID":"1e3c24de-f263-4f2d-a265-cb2ea4a61dff","Type":"ContainerStarted","Data":"feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab"}
Sep 10 14:31:46.963295 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.963244    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:46.978921 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.978871    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" podStartSLOduration=1.978856579 podStartE2EDuration="1.978856579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:46.977552233 +0000 UTC m=+1218.939061752" watchObservedRunningTime="2026-09-10 14:31:46.978856579 +0000 UTC m=+1218.940366096"
Sep 10 14:31:47.000742 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:46.999849    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" podStartSLOduration=2.5561040779999997 podStartE2EDuration="2.999828719s" totalImagesPullingTime="443.724641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:46.994847315 +0000 UTC m=+1218.956356833" watchObservedRunningTime="2026-09-10 14:31:46.999828719 +0000 UTC m=+1218.961338239"
Sep 10 14:31:52.974058 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:52.974028    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:31:57.969869 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:31:57.969839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:32:13.030948 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.030917    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:32:13.042250 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.042222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.045224 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.045192    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-9dcc39-serving-cert\""
Sep 10 14:32:13.052107 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.052082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:32:13.161717 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.161666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.161889 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.161763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfgbb\" (UniqueName: \"kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.161889 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.161791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.161966 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.161915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.262605 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.262566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.262834 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.262619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.262834 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.262657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfgbb\" (UniqueName: \"kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.262834 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.262674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.263315 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.263290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.265275 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.265245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.265375 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.265245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.276009 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.275984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfgbb\" (UniqueName: \"kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb\") pod \"maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.355458 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.355385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:13.632475 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.632447    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:32:13.634433 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:32:13.634403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acb7831_46a3_44db_af96_34f39036666f.slice/crio-a892d4caed6ee97501bbd9c74c6667da5eae3ddcd369fd027f325ae50f0a5027 WatchSource:0}: Error finding container a892d4caed6ee97501bbd9c74c6667da5eae3ddcd369fd027f325ae50f0a5027: Status 404 returned error can't find the container with id a892d4caed6ee97501bbd9c74c6667da5eae3ddcd369fd027f325ae50f0a5027
Sep 10 14:32:13.804276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.804235    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:32:13.807798 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.807773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:13.810750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.810726    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-9dcc39\""
Sep 10 14:32:13.820662 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.820640    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:32:13.969641 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.969552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f2fg\" (UniqueName: \"kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:13.969641 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:13.969622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.070817 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.070791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f2fg\" (UniqueName: \"kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.071161 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.070852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.071481 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.071463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.080331 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.080312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f2fg\" (UniqueName: \"kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg\") pod \"payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.096867 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.096840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" event={"ID":"6acb7831-46a3-44db-af96-34f39036666f","Type":"ContainerStarted","Data":"a892d4caed6ee97501bbd9c74c6667da5eae3ddcd369fd027f325ae50f0a5027"}
Sep 10 14:32:14.118304 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.118283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:14.396606 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:14.396574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:32:14.397771 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:32:14.397747    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bad7d32_e805_4aef_a751_b696ca738b68.slice/crio-58ca5b1a4ab676037b58740fd82feff7e32e698cc45cbc76faff89bf950c2dc2 WatchSource:0}: Error finding container 58ca5b1a4ab676037b58740fd82feff7e32e698cc45cbc76faff89bf950c2dc2: Status 404 returned error can't find the container with id 58ca5b1a4ab676037b58740fd82feff7e32e698cc45cbc76faff89bf950c2dc2
Sep 10 14:32:15.103104 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.103068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" event={"ID":"9bad7d32-e805-4aef-a751-b696ca738b68","Type":"ContainerStarted","Data":"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"}
Sep 10 14:32:15.103564 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.103119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" event={"ID":"9bad7d32-e805-4aef-a751-b696ca738b68","Type":"ContainerStarted","Data":"58ca5b1a4ab676037b58740fd82feff7e32e698cc45cbc76faff89bf950c2dc2"}
Sep 10 14:32:15.103564 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.103186    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:15.104807 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.104787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" event={"ID":"6acb7831-46a3-44db-af96-34f39036666f","Type":"ContainerStarted","Data":"a1d3174b9897233133566e0d10e5eff0854f393028d02446847109bb52888247"}
Sep 10 14:32:15.104896 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.104843    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:15.130309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.130258    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" podStartSLOduration=2.130244358 podStartE2EDuration="2.130244358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:15.129003824 +0000 UTC m=+1247.090513343" watchObservedRunningTime="2026-09-10 14:32:15.130244358 +0000 UTC m=+1247.091753904"
Sep 10 14:32:15.154259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:15.154189    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" podStartSLOduration=1.7298063190000001 podStartE2EDuration="2.154173178s" totalImagesPullingTime="424.366859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:15.151956859 +0000 UTC m=+1247.113466376" watchObservedRunningTime="2026-09-10 14:32:15.154173178 +0000 UTC m=+1247.115682696"
Sep 10 14:32:18.041990 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.041950    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:32:18.044936 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.044919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.047971 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.047950    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-54c040\""
Sep 10 14:32:18.048109 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.048086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-54c040-dockercfg-ghj55\""
Sep 10 14:32:18.056045 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.056021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:32:18.211924 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.211881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.212256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.212232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxknq\" (UniqueName: \"kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.313076 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.312982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.313359 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.313075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxknq\" (UniqueName: \"kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.313809 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.313782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.321909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.321883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxknq\" (UniqueName: \"kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq\") pod \"payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.357016 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.356981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:18.679175 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:18.679143    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:32:18.681461 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:32:18.681381    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb0b70a2_387b_42a3_8eb6_17bf4cadf88e.slice/crio-cffbeb0ce62e364eecb734025a791d7b929d82bc698c76146e0518562091b369 WatchSource:0}: Error finding container cffbeb0ce62e364eecb734025a791d7b929d82bc698c76146e0518562091b369: Status 404 returned error can't find the container with id cffbeb0ce62e364eecb734025a791d7b929d82bc698c76146e0518562091b369
Sep 10 14:32:19.125680 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:19.125640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" event={"ID":"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e","Type":"ContainerStarted","Data":"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"}
Sep 10 14:32:19.126096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:19.125713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" event={"ID":"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e","Type":"ContainerStarted","Data":"cffbeb0ce62e364eecb734025a791d7b929d82bc698c76146e0518562091b369"}
Sep 10 14:32:19.126096 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:19.125740    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:19.143322 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:19.143275    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" podStartSLOduration=1.143260063 podStartE2EDuration="1.143260063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:19.14140108 +0000 UTC m=+1251.102910597" watchObservedRunningTime="2026-09-10 14:32:19.143260063 +0000 UTC m=+1251.104769581"
Sep 10 14:32:21.116132 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:21.116053    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:32:26.112793 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:26.112764    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:32:30.133889 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:30.133854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:32:55.137024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.136992    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:32:55.141745 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.141691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.144979 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.144957    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e96075f7-serving-cert\""
Sep 10 14:32:55.154450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.154427    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:32:55.242104 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.242066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.242294 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.242189    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9lfz\" (UniqueName: \"kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.242294 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.242271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.242394 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.242341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.343476 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.343441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9lfz\" (UniqueName: \"kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.343696 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.343534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.343696 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.343573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.343696 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.343604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.344267 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.344243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.346040 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.346019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.346138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.346058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.352868 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.352841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9lfz\" (UniqueName: \"kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz\") pod \"maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.453751 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.453633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:55.726651 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.726624    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:32:55.728651 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:32:55.728620    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7861f66_a68f_4e4b_a3c8_cf9274e83eb7.slice/crio-0ca2a7e82c53e6dc48bc69ac7c80afffcce46f8f09d44fd9f3d76ef7dfb9b00b WatchSource:0}: Error finding container 0ca2a7e82c53e6dc48bc69ac7c80afffcce46f8f09d44fd9f3d76ef7dfb9b00b: Status 404 returned error can't find the container with id 0ca2a7e82c53e6dc48bc69ac7c80afffcce46f8f09d44fd9f3d76ef7dfb9b00b
Sep 10 14:32:55.821436 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.821394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:32:55.826196 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.826173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:55.829022 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.829001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e96075f7\""
Sep 10 14:32:55.829172 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.829154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e96075f7-dockercfg-jqjpw\""
Sep 10 14:32:55.837483 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.837453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:32:55.949438 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.949396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:55.949641 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:55.949604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsrkg\" (UniqueName: \"kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.050241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.050206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsrkg\" (UniqueName: \"kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.050441 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.050257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.050953 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.050931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.058746 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.058723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsrkg\" (UniqueName: \"kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg\") pod \"payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") " pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.138292 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.138251    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:56.309492 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.309455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" event={"ID":"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7","Type":"ContainerStarted","Data":"0ca2a7e82c53e6dc48bc69ac7c80afffcce46f8f09d44fd9f3d76ef7dfb9b00b"}
Sep 10 14:32:56.412540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:56.412515    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:32:56.414061 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:32:56.414034    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4a6ad4_e539_4b6d_b1de_463ec3b98305.slice/crio-d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151 WatchSource:0}: Error finding container d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151: Status 404 returned error can't find the container with id d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151
Sep 10 14:32:57.317399 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.317360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" event={"ID":"4f4a6ad4-e539-4b6d-b1de-463ec3b98305","Type":"ContainerStarted","Data":"3bbb66b51d323bb71ec6c79f30382a9e18934be0fe4681ea59a5bd4195ec3702"}
Sep 10 14:32:57.317900 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.317423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" event={"ID":"4f4a6ad4-e539-4b6d-b1de-463ec3b98305","Type":"ContainerStarted","Data":"d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151"}
Sep 10 14:32:57.317900 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.317559    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:32:57.319004 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.318981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" event={"ID":"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7","Type":"ContainerStarted","Data":"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"}
Sep 10 14:32:57.319130 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.319090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:32:57.335609 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.335559    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" podStartSLOduration=2.335541747 podStartE2EDuration="2.335541747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:57.334307925 +0000 UTC m=+1289.295817445" watchObservedRunningTime="2026-09-10 14:32:57.335541747 +0000 UTC m=+1289.297051267"
Sep 10 14:32:57.353243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:32:57.353202    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" podStartSLOduration=1.8210265159999999 podStartE2EDuration="2.353187446s" totalImagesPullingTime="532.16093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:57.352024901 +0000 UTC m=+1289.313534420" watchObservedRunningTime="2026-09-10 14:32:57.353187446 +0000 UTC m=+1289.314697009"
Sep 10 14:33:03.329862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:03.329834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:33:08.326049 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:08.326017    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:33:30.393348 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.393312    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:33:30.397078 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.397054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.399728 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.399681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"auth-test-model-9dcc39-kserve-self-signed-certs\""
Sep 10 14:33:30.401032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.401001    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"default-dockercfg-442hr\""
Sep 10 14:33:30.401184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.401093    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"kube-root-ca.crt\""
Sep 10 14:33:30.401184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.401001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"openshift-service-ca.crt\""
Sep 10 14:33:30.408292 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.408269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:33:30.482757 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.482719    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.482944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.482786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn8nv\" (UniqueName: \"kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.482944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.482876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.482944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.482933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.483115 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.482987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.483115 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.483032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584040 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.583994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584040 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn8nv\" (UniqueName: \"kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584488 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.584593 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.584570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.586572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.586555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.586710 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.586684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.592850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.592828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn8nv\" (UniqueName: \"kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv\") pod \"auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.707264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.707178    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:30.985101 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:30.985019    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:33:30.987993 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:33:30.987950    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod069d893b_2756_401c_9c24_c96d291efe1c.slice/crio-8ef1b156f537acb9ccd9bec075cd705f37108f04224c6967719003317d9f6f83 WatchSource:0}: Error finding container 8ef1b156f537acb9ccd9bec075cd705f37108f04224c6967719003317d9f6f83: Status 404 returned error can't find the container with id 8ef1b156f537acb9ccd9bec075cd705f37108f04224c6967719003317d9f6f83
Sep 10 14:33:31.499890 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:31.499840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" event={"ID":"069d893b-2756-401c-9c24-c96d291efe1c","Type":"ContainerStarted","Data":"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"}
Sep 10 14:33:31.499890 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:31.499891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" event={"ID":"069d893b-2756-401c-9c24-c96d291efe1c","Type":"ContainerStarted","Data":"8ef1b156f537acb9ccd9bec075cd705f37108f04224c6967719003317d9f6f83"}
Sep 10 14:33:31.524276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:31.524229    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" podStartSLOduration=1.524214554 podStartE2EDuration="1.524214554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:31.522568396 +0000 UTC m=+1323.484077914" watchObservedRunningTime="2026-09-10 14:33:31.524214554 +0000 UTC m=+1323.485724071"
Sep 10 14:33:40.707618 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:40.707582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:40.708034 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:40.707637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:40.720162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:40.720135    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:33:41.560855 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:33:41.560821    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:34:44.540385 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.540352    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:34:44.541034 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.540583    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" podUID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" containerName="maas-api" containerID="cri-o://de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8" gracePeriod=30
Sep 10 14:34:44.599450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.599420    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:34:44.599658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.599620    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" podUID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" containerName="payload-processing" containerID="cri-o://3bbb66b51d323bb71ec6c79f30382a9e18934be0fe4681ea59a5bd4195ec3702" gracePeriod=30
Sep 10 14:34:44.846591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.846562    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:34:44.871043 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.871015    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" containerID="de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8" exitCode=0
Sep 10 14:34:44.871170 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.871058    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"
Sep 10 14:34:44.871170 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.871095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" event={"ID":"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7","Type":"ContainerDied","Data":"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"}
Sep 10 14:34:44.871170 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.871154    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn" event={"ID":"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7","Type":"ContainerDied","Data":"0ca2a7e82c53e6dc48bc69ac7c80afffcce46f8f09d44fd9f3d76ef7dfb9b00b"}
Sep 10 14:34:44.871328 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.871181    2581 scope.go:117] "RemoveContainer" containerID="de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"
Sep 10 14:34:44.872717 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.872679    2581 generic.go:356] "Generic (PLEG): container finished" podID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" containerID="3bbb66b51d323bb71ec6c79f30382a9e18934be0fe4681ea59a5bd4195ec3702" exitCode=0
Sep 10 14:34:44.872811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.872739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" event={"ID":"4f4a6ad4-e539-4b6d-b1de-463ec3b98305","Type":"ContainerDied","Data":"3bbb66b51d323bb71ec6c79f30382a9e18934be0fe4681ea59a5bd4195ec3702"}
Sep 10 14:34:44.872811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.872789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk" event={"ID":"4f4a6ad4-e539-4b6d-b1de-463ec3b98305","Type":"ContainerDied","Data":"d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151"}
Sep 10 14:34:44.872811 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.872807    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d176150ae8d1decdc410ed2665c4375f873ec49a8449abd349d73a3cb08e3151"
Sep 10 14:34:44.880564 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.880457    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:34:44.883517 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.883492    2581 scope.go:117] "RemoveContainer" containerID="de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"
Sep 10 14:34:44.883827 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:34:44.883805    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8\": container with ID starting with de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8 not found: ID does not exist" containerID="de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"
Sep 10 14:34:44.883917 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.883834    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8"} err="failed to get container status \"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8\": rpc error: code = NotFound desc = could not find container \"de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8\": container with ID starting with de34594185297bbbf9989343b812ea65ed97b491eaa37dc3962aa4909049fbb8 not found: ID does not exist"
Sep 10 14:34:44.918251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918224    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9lfz\" (UniqueName: \"kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz\") pod \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume\") pod \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca\") pod \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsrkg\" (UniqueName: \"kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg\") pod \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\" (UID: \"4f4a6ad4-e539-4b6d-b1de-463ec3b98305\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs\") pod \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls\") pod \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\" (UID: \"a7861f66-a68f-4e4b-a3c8-cf9274e83eb7\") "
Sep 10 14:34:44.919080 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.918663    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f4a6ad4-e539-4b6d-b1de-463ec3b98305" (UID: "4f4a6ad4-e539-4b6d-b1de-463ec3b98305"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:34:44.919523 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.919150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" (UID: "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:34:44.919583 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.919524    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:44.923470 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.922304    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz" (OuterVolumeSpecName: "kube-api-access-k9lfz") pod "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" (UID: "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7"). InnerVolumeSpecName "kube-api-access-k9lfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:44.923470 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.922653    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" (UID: "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:34:44.923470 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.923193    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg" (OuterVolumeSpecName: "kube-api-access-dsrkg") pod "4f4a6ad4-e539-4b6d-b1de-463ec3b98305" (UID: "4f4a6ad4-e539-4b6d-b1de-463ec3b98305"). InnerVolumeSpecName "kube-api-access-dsrkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:44.924032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:44.924008    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" (UID: "a7861f66-a68f-4e4b-a3c8-cf9274e83eb7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:34:45.020109 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.020081    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9lfz\" (UniqueName: \"kubernetes.io/projected/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-kube-api-access-k9lfz\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.020211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.020109    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.020211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.020123    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsrkg\" (UniqueName: \"kubernetes.io/projected/4f4a6ad4-e539-4b6d-b1de-463ec3b98305-kube-api-access-dsrkg\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.020211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.020136    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.020211 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.020150    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.192881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.192855    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:34:45.196903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.196881    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e96075f7-79d7b6fc55-4dnvn"]
Sep 10 14:34:45.883129 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.883044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"
Sep 10 14:34:45.904975 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.904947    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:34:45.906741 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:45.906718    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e96075f7-7ff67549f7-4c6fk"]
Sep 10 14:34:46.532440 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:46.532407    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" path="/var/lib/kubelet/pods/4f4a6ad4-e539-4b6d-b1de-463ec3b98305/volumes"
Sep 10 14:34:46.532819 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:34:46.532805    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" path="/var/lib/kubelet/pods/a7861f66-a68f-4e4b-a3c8-cf9274e83eb7/volumes"
Sep 10 14:35:47.135904 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:35:47.135868    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:36:02.155883 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.155820    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" podUID="069d893b-2756-401c-9c24-c96d291efe1c" containerName="main" containerID="cri-o://a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a" gracePeriod=45
Sep 10 14:36:02.393774 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.393743    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:36:02.509156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509371 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509371 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509334    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509371 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509359    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509371 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:02.509597 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn8nv\" (UniqueName: \"kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv\") pod \"069d893b-2756-401c-9c24-c96d291efe1c\" (UID: \"069d893b-2756-401c-9c24-c96d291efe1c\") "
Sep 10 14:36:02.509597 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509500    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:02.509688 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home" (OuterVolumeSpecName: "home") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:02.509763 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509749    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-tmp-dir\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.509829 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509763    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-home\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.509829 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.509771    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-model-cache\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.511435 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.511404    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:36:02.511435 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.511418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm" (OuterVolumeSpecName: "dshm") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:02.511435 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.511418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv" (OuterVolumeSpecName: "kube-api-access-pn8nv") pod "069d893b-2756-401c-9c24-c96d291efe1c" (UID: "069d893b-2756-401c-9c24-c96d291efe1c"). InnerVolumeSpecName "kube-api-access-pn8nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:36:02.611002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.610965    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/069d893b-2756-401c-9c24-c96d291efe1c-tls-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.611002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.610995    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/069d893b-2756-401c-9c24-c96d291efe1c-dshm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.611002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:02.611004    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn8nv\" (UniqueName: \"kubernetes.io/projected/069d893b-2756-401c-9c24-c96d291efe1c-kube-api-access-pn8nv\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:36:03.259013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.258976    2581 generic.go:356] "Generic (PLEG): container finished" podID="069d893b-2756-401c-9c24-c96d291efe1c" containerID="a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a" exitCode=0
Sep 10 14:36:03.259450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.259025    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"
Sep 10 14:36:03.259450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.259057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" event={"ID":"069d893b-2756-401c-9c24-c96d291efe1c","Type":"ContainerDied","Data":"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"}
Sep 10 14:36:03.259450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.259101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd" event={"ID":"069d893b-2756-401c-9c24-c96d291efe1c","Type":"ContainerDied","Data":"8ef1b156f537acb9ccd9bec075cd705f37108f04224c6967719003317d9f6f83"}
Sep 10 14:36:03.259450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.259113    2581 scope.go:117] "RemoveContainer" containerID="a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"
Sep 10 14:36:03.267772 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.267755    2581 scope.go:117] "RemoveContainer" containerID="a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"
Sep 10 14:36:03.268042 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:36:03.268024    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a\": container with ID starting with a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a not found: ID does not exist" containerID="a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"
Sep 10 14:36:03.268088 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.268049    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a"} err="failed to get container status \"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a\": rpc error: code = NotFound desc = could not find container \"a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a\": container with ID starting with a7ddb9a39c3a0eed8078c82d6924d0b47c7d89168d9a52158b04a6a52155c30a not found: ID does not exist"
Sep 10 14:36:03.274903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.274881    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:36:03.278187 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:03.278165    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/auth-test-model-9dcc39-kserve-dddc78b4c-8zrfd"]
Sep 10 14:36:04.532232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:04.532197    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="069d893b-2756-401c-9c24-c96d291efe1c" path="/var/lib/kubelet/pods/069d893b-2756-401c-9c24-c96d291efe1c/volumes"
Sep 10 14:36:28.551427 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:28.551398    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:36:28.555991 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:28.555970    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:36:34.827794 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.827760    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828163    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" containerName="maas-api"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828177    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" containerName="maas-api"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828202    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="069d893b-2756-401c-9c24-c96d291efe1c" containerName="main"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828209    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="069d893b-2756-401c-9c24-c96d291efe1c" containerName="main"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828230    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" containerName="payload-processing"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828238    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" containerName="payload-processing"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828299    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4a6ad4-e539-4b6d-b1de-463ec3b98305" containerName="payload-processing"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828309    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7861f66-a68f-4e4b-a3c8-cf9274e83eb7" containerName="maas-api"
Sep 10 14:36:34.828325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.828321    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="069d893b-2756-401c-9c24-c96d291efe1c" containerName="main"
Sep 10 14:36:34.831371 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.831355    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:34.833931 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.833904    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-53471f-serving-cert\""
Sep 10 14:36:34.843494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.843473    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:36:34.920426 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.920395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:34.920596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.920455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9cx\" (UniqueName: \"kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:34.920596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.920556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:34.920596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:34.920592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.021619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.021584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.021619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.021625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.021892 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.021661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.021892 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.021694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9cx\" (UniqueName: \"kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.021892 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:36:35.021770    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-serving-cert: secret "maas-api-e2e-api-a-53471f-serving-cert" not found
Sep 10 14:36:35.021892 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:36:35.021843    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls podName:a5dbeeb6-2034-4fe8-acd6-e226ec7847f2 nodeName:}" failed. No retries permitted until 2026-09-10 14:36:35.521820927 +0000 UTC m=+1507.483330426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls") pod "maas-api-e2e-api-a-53471f-74f87d544c-z5scz" (UID: "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2") : secret "maas-api-e2e-api-a-53471f-serving-cert" not found
Sep 10 14:36:35.022356 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.022332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.024361 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.024339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.032520 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.032484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9cx\" (UniqueName: \"kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.522235 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.522196    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:36:35.525903 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.525882    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.526325 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.526304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.528549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.528522    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-53471f-dockercfg-qhwnb\""
Sep 10 14:36:35.528549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.528529    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-53471f\""
Sep 10 14:36:35.528826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.528806    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") pod \"maas-api-e2e-api-a-53471f-74f87d544c-z5scz\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.534393 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.534372    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:36:35.627804 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.627770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.628012 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.627842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69db\" (UniqueName: \"kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.728893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.728855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.729063 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.728936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69db\" (UniqueName: \"kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.729548 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.729530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.741132 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.741110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:35.744489 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.744205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69db\" (UniqueName: \"kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db\") pod \"payload-processing-e2e-api-a-53471f-5689c77947-5d2jd\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") " pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:35.845943 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:35.845912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:36.015392 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.015365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:36:36.017151 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:36:36.017124    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5dbeeb6_2034_4fe8_acd6_e226ec7847f2.slice/crio-2115a12e26e97de454681da2efc1ae15d51d409dfe131d0fdd68362e20eca750 WatchSource:0}: Error finding container 2115a12e26e97de454681da2efc1ae15d51d409dfe131d0fdd68362e20eca750: Status 404 returned error can't find the container with id 2115a12e26e97de454681da2efc1ae15d51d409dfe131d0fdd68362e20eca750
Sep 10 14:36:36.018504 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.018485    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:36:36.118079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.118051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:36:36.120752 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:36:36.120723    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f95c10_55a9_4b25_b875_3bec284bc3dc.slice/crio-e22ce9dc78417634cc4f8ab59e81034e993fd366b4e9089f193b6a4f88eb7d4f WatchSource:0}: Error finding container e22ce9dc78417634cc4f8ab59e81034e993fd366b4e9089f193b6a4f88eb7d4f: Status 404 returned error can't find the container with id e22ce9dc78417634cc4f8ab59e81034e993fd366b4e9089f193b6a4f88eb7d4f
Sep 10 14:36:36.423507 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.423411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" event={"ID":"03f95c10-55a9-4b25-b875-3bec284bc3dc","Type":"ContainerStarted","Data":"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"}
Sep 10 14:36:36.423507 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.423487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" event={"ID":"03f95c10-55a9-4b25-b875-3bec284bc3dc","Type":"ContainerStarted","Data":"e22ce9dc78417634cc4f8ab59e81034e993fd366b4e9089f193b6a4f88eb7d4f"}
Sep 10 14:36:36.423772 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.423633    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:36:36.424766 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.424744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" event={"ID":"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2","Type":"ContainerStarted","Data":"2115a12e26e97de454681da2efc1ae15d51d409dfe131d0fdd68362e20eca750"}
Sep 10 14:36:36.443254 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:36.443213    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" podStartSLOduration=1.4432002179999999 podStartE2EDuration="1.443200218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:36.441895878 +0000 UTC m=+1508.403405396" watchObservedRunningTime="2026-09-10 14:36:36.443200218 +0000 UTC m=+1508.404709771"
Sep 10 14:36:37.431031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:37.430989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" event={"ID":"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2","Type":"ContainerStarted","Data":"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"}
Sep 10 14:36:37.431526 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:37.431161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:37.451192 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:37.451145    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" podStartSLOduration=2.984633617 podStartE2EDuration="3.451131474s" totalImagesPullingTime="466.497857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:37.448523859 +0000 UTC m=+1509.410033378" watchObservedRunningTime="2026-09-10 14:36:37.451131474 +0000 UTC m=+1509.412640993"
Sep 10 14:36:43.441932 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:43.441903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:36:47.432866 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:36:47.432834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:37:03.454472 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.454393    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:37:03.458517 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.458484    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.461266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.461243    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7ae247-serving-cert\""
Sep 10 14:37:03.470862 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.470835    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:37:03.586431 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.586393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.586431 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.586437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.586658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.586466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kxrw\" (UniqueName: \"kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.586658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.586528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.687406 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.687370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.687601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.687420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.687601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.687463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kxrw\" (UniqueName: \"kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.687601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.687546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.687601 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:37:03.687585    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-serving-cert: secret "maas-api-e2e-api-b-7ae247-serving-cert" not found
Sep 10 14:37:03.687901 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:37:03.687671    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls podName:01b52b8e-935f-446b-a232-36a6672373a0 nodeName:}" failed. No retries permitted until 2026-09-10 14:37:04.187651228 +0000 UTC m=+1536.149160730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls") pod "maas-api-e2e-api-b-7ae247-649859846-d75gt" (UID: "01b52b8e-935f-446b-a232-36a6672373a0") : secret "maas-api-e2e-api-b-7ae247-serving-cert" not found
Sep 10 14:37:03.688166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.688148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.690156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.690132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:03.706332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:03.706272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kxrw\" (UniqueName: \"kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:04.192200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:04.192165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:04.194499 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:04.194473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") pod \"maas-api-e2e-api-b-7ae247-649859846-d75gt\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:04.371892 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:04.371855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:04.643352 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:04.643321    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:37:04.645058 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:37:04.645033    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b52b8e_935f_446b_a232_36a6672373a0.slice/crio-951ad80b3b8f5e22deaa13a8948e484cdd4fb12cc96f64422f2e296d777991de WatchSource:0}: Error finding container 951ad80b3b8f5e22deaa13a8948e484cdd4fb12cc96f64422f2e296d777991de: Status 404 returned error can't find the container with id 951ad80b3b8f5e22deaa13a8948e484cdd4fb12cc96f64422f2e296d777991de
Sep 10 14:37:05.570102 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:05.570073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" event={"ID":"01b52b8e-935f-446b-a232-36a6672373a0","Type":"ContainerStarted","Data":"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"}
Sep 10 14:37:05.570337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:05.570124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" event={"ID":"01b52b8e-935f-446b-a232-36a6672373a0","Type":"ContainerStarted","Data":"951ad80b3b8f5e22deaa13a8948e484cdd4fb12cc96f64422f2e296d777991de"}
Sep 10 14:37:05.570337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:05.570214    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:05.589007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:05.588956    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" podStartSLOduration=2.205187161 podStartE2EDuration="2.58894151s" totalImagesPullingTime="383.754349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:05.586567243 +0000 UTC m=+1537.548076764" watchObservedRunningTime="2026-09-10 14:37:05.58894151 +0000 UTC m=+1537.550451028"
Sep 10 14:37:11.580384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:11.580352    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:37:45.699193 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:45.699159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:37:45.699789 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:45.699415    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" podUID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" containerName="maas-api" containerID="cri-o://3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75" gracePeriod=30
Sep 10 14:37:45.790393 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:45.790359    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:37:45.790626 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:45.790596    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" podUID="03f95c10-55a9-4b25-b875-3bec284bc3dc" containerName="payload-processing" containerID="cri-o://f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a" gracePeriod=30
Sep 10 14:37:46.035816 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.035795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:37:46.065021 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.064998    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:37:46.150353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") pod \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") "
Sep 10 14:37:46.150494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150363    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb9cx\" (UniqueName: \"kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx\") pod \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") "
Sep 10 14:37:46.150494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150405    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume\") pod \"03f95c10-55a9-4b25-b875-3bec284bc3dc\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") "
Sep 10 14:37:46.150494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs\") pod \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") "
Sep 10 14:37:46.150494 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p69db\" (UniqueName: \"kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db\") pod \"03f95c10-55a9-4b25-b875-3bec284bc3dc\" (UID: \"03f95c10-55a9-4b25-b875-3bec284bc3dc\") "
Sep 10 14:37:46.150737 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150528    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca\") pod \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\" (UID: \"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2\") "
Sep 10 14:37:46.150919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03f95c10-55a9-4b25-b875-3bec284bc3dc" (UID: "03f95c10-55a9-4b25-b875-3bec284bc3dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:37:46.151002 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.150917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" (UID: "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:37:46.152591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.152564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" (UID: "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:37:46.152687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.152619    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db" (OuterVolumeSpecName: "kube-api-access-p69db") pod "03f95c10-55a9-4b25-b875-3bec284bc3dc" (UID: "03f95c10-55a9-4b25-b875-3bec284bc3dc"). InnerVolumeSpecName "kube-api-access-p69db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:46.152687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.152639    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" (UID: "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:37:46.152687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.152642    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx" (OuterVolumeSpecName: "kube-api-access-wb9cx") pod "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" (UID: "a5dbeeb6-2034-4fe8-acd6-e226ec7847f2"). InnerVolumeSpecName "kube-api-access-wb9cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:46.252059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.251995    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.252059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.252019    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.252059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.252029    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb9cx\" (UniqueName: \"kubernetes.io/projected/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-kube-api-access-wb9cx\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.252059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.252041    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03f95c10-55a9-4b25-b875-3bec284bc3dc-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.252059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.252053    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.252283 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.252066    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p69db\" (UniqueName: \"kubernetes.io/projected/03f95c10-55a9-4b25-b875-3bec284bc3dc-kube-api-access-p69db\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:37:46.777797 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.777761    2581 generic.go:356] "Generic (PLEG): container finished" podID="03f95c10-55a9-4b25-b875-3bec284bc3dc" containerID="f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a" exitCode=0
Sep 10 14:37:46.778241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.777815    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"
Sep 10 14:37:46.778241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.777851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" event={"ID":"03f95c10-55a9-4b25-b875-3bec284bc3dc","Type":"ContainerDied","Data":"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"}
Sep 10 14:37:46.778241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.777922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd" event={"ID":"03f95c10-55a9-4b25-b875-3bec284bc3dc","Type":"ContainerDied","Data":"e22ce9dc78417634cc4f8ab59e81034e993fd366b4e9089f193b6a4f88eb7d4f"}
Sep 10 14:37:46.778241 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.777947    2581 scope.go:117] "RemoveContainer" containerID="f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"
Sep 10 14:37:46.779355 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.779333    2581 generic.go:356] "Generic (PLEG): container finished" podID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" containerID="3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75" exitCode=0
Sep 10 14:37:46.779456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.779374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" event={"ID":"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2","Type":"ContainerDied","Data":"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"}
Sep 10 14:37:46.779456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.779387    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"
Sep 10 14:37:46.779456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.779417    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz" event={"ID":"a5dbeeb6-2034-4fe8-acd6-e226ec7847f2","Type":"ContainerDied","Data":"2115a12e26e97de454681da2efc1ae15d51d409dfe131d0fdd68362e20eca750"}
Sep 10 14:37:46.786963 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.786946    2581 scope.go:117] "RemoveContainer" containerID="f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"
Sep 10 14:37:46.787243 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:37:46.787201    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a\": container with ID starting with f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a not found: ID does not exist" containerID="f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"
Sep 10 14:37:46.787243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.787228    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a"} err="failed to get container status \"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a\": rpc error: code = NotFound desc = could not find container \"f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a\": container with ID starting with f75db1d286a11564213db55ea1731b4e5dc6e63c0342aaadb7cb97af1250fd5a not found: ID does not exist"
Sep 10 14:37:46.787243 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.787243    2581 scope.go:117] "RemoveContainer" containerID="3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"
Sep 10 14:37:46.798504 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.798484    2581 scope.go:117] "RemoveContainer" containerID="3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"
Sep 10 14:37:46.799022 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:37:46.798998    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75\": container with ID starting with 3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75 not found: ID does not exist" containerID="3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"
Sep 10 14:37:46.799122 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.799032    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75"} err="failed to get container status \"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75\": rpc error: code = NotFound desc = could not find container \"3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75\": container with ID starting with 3822058100fc66daa15c741f3a22f75857547ffe61f753cf6511d7c8042cbd75 not found: ID does not exist"
Sep 10 14:37:46.800897 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.800877    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:37:46.802666 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.802648    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53471f-5689c77947-5d2jd"]
Sep 10 14:37:46.814235 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.814214    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:37:46.816415 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:46.816394    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53471f-74f87d544c-z5scz"]
Sep 10 14:37:48.532307 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:48.532277    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f95c10-55a9-4b25-b875-3bec284bc3dc" path="/var/lib/kubelet/pods/03f95c10-55a9-4b25-b875-3bec284bc3dc/volumes"
Sep 10 14:37:48.532671 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:37:48.532643    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" path="/var/lib/kubelet/pods/a5dbeeb6-2034-4fe8-acd6-e226ec7847f2/volumes"
Sep 10 14:38:08.071194 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.071160    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:38:08.071612 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.071408    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" podUID="01b52b8e-935f-446b-a232-36a6672373a0" containerName="maas-api" containerID="cri-o://2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7" gracePeriod=30
Sep 10 14:38:08.317407 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.317384    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:38:08.342314 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.342250    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kxrw\" (UniqueName: \"kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw\") pod \"01b52b8e-935f-446b-a232-36a6672373a0\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") "
Sep 10 14:38:08.342314 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.342290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") pod \"01b52b8e-935f-446b-a232-36a6672373a0\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") "
Sep 10 14:38:08.342491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.342333    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca\") pod \"01b52b8e-935f-446b-a232-36a6672373a0\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") "
Sep 10 14:38:08.342491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.342438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs\") pod \"01b52b8e-935f-446b-a232-36a6672373a0\" (UID: \"01b52b8e-935f-446b-a232-36a6672373a0\") "
Sep 10 14:38:08.342855 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.342824    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01b52b8e-935f-446b-a232-36a6672373a0" (UID: "01b52b8e-935f-446b-a232-36a6672373a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:38:08.344384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.344355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01b52b8e-935f-446b-a232-36a6672373a0" (UID: "01b52b8e-935f-446b-a232-36a6672373a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:08.344480 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.344394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw" (OuterVolumeSpecName: "kube-api-access-5kxrw") pod "01b52b8e-935f-446b-a232-36a6672373a0" (UID: "01b52b8e-935f-446b-a232-36a6672373a0"). InnerVolumeSpecName "kube-api-access-5kxrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:08.344480 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.344427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "01b52b8e-935f-446b-a232-36a6672373a0" (UID: "01b52b8e-935f-446b-a232-36a6672373a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:08.443635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.443609    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kxrw\" (UniqueName: \"kubernetes.io/projected/01b52b8e-935f-446b-a232-36a6672373a0-kube-api-access-5kxrw\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:08.443635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.443633    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:08.443801 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.443642    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b52b8e-935f-446b-a232-36a6672373a0-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:08.443801 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.443650    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b52b8e-935f-446b-a232-36a6672373a0-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:08.891863 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.891830    2581 generic.go:356] "Generic (PLEG): container finished" podID="01b52b8e-935f-446b-a232-36a6672373a0" containerID="2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7" exitCode=0
Sep 10 14:38:08.892056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.891872    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"
Sep 10 14:38:08.892056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.891892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" event={"ID":"01b52b8e-935f-446b-a232-36a6672373a0","Type":"ContainerDied","Data":"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"}
Sep 10 14:38:08.892056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.891939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt" event={"ID":"01b52b8e-935f-446b-a232-36a6672373a0","Type":"ContainerDied","Data":"951ad80b3b8f5e22deaa13a8948e484cdd4fb12cc96f64422f2e296d777991de"}
Sep 10 14:38:08.892056 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.891962    2581 scope.go:117] "RemoveContainer" containerID="2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"
Sep 10 14:38:08.900562 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.900545    2581 scope.go:117] "RemoveContainer" containerID="2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"
Sep 10 14:38:08.900835 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:38:08.900812    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7\": container with ID starting with 2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7 not found: ID does not exist" containerID="2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"
Sep 10 14:38:08.900901 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.900843    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7"} err="failed to get container status \"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7\": rpc error: code = NotFound desc = could not find container \"2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7\": container with ID starting with 2a2bf09a2ab19c344d2ed49eeeead5e237ff8cfb569a476a12c4a499382949f7 not found: ID does not exist"
Sep 10 14:38:08.910519 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.910496    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:38:08.914499 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:08.914478    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7ae247-649859846-d75gt"]
Sep 10 14:38:10.531437 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:10.531403    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b52b8e-935f-446b-a232-36a6672373a0" path="/var/lib/kubelet/pods/01b52b8e-935f-446b-a232-36a6672373a0/volumes"
Sep 10 14:38:11.404186 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404150    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:38:11.404561 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404548    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f95c10-55a9-4b25-b875-3bec284bc3dc" containerName="payload-processing"
Sep 10 14:38:11.404616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404563    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f95c10-55a9-4b25-b875-3bec284bc3dc" containerName="payload-processing"
Sep 10 14:38:11.404616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404571    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b52b8e-935f-446b-a232-36a6672373a0" containerName="maas-api"
Sep 10 14:38:11.404616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404577    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b52b8e-935f-446b-a232-36a6672373a0" containerName="maas-api"
Sep 10 14:38:11.404616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404587    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" containerName="maas-api"
Sep 10 14:38:11.404616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404592    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" containerName="maas-api"
Sep 10 14:38:11.404797 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404664    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b52b8e-935f-446b-a232-36a6672373a0" containerName="maas-api"
Sep 10 14:38:11.404797 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404673    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5dbeeb6-2034-4fe8-acd6-e226ec7847f2" containerName="maas-api"
Sep 10 14:38:11.404797 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.404682    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f95c10-55a9-4b25-b875-3bec284bc3dc" containerName="payload-processing"
Sep 10 14:38:11.408887 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.408871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.411288 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.411267    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"default-dockercfg-442hr\""
Sep 10 14:38:11.411420 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.411330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"openshift-service-ca.crt\""
Sep 10 14:38:11.412511 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.412493    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"rate-test-model-9dcc39-kserve-self-signed-certs\""
Sep 10 14:38:11.412597 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.412535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"kube-root-ca.crt\""
Sep 10 14:38:11.417672 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.417649    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:38:11.467410 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.467563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.467563 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.467658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.467727 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.467727 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.467692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsd7r\" (UniqueName: \"kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568141 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568590 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsd7r\" (UniqueName: \"kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568973 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.568973 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.569068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.568978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.570384 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.570362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.570826 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.570806    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.576562 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.576541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsd7r\" (UniqueName: \"kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r\") pod \"rate-test-model-9dcc39-kserve-756f748847-vxfk4\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.718588 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.718518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:11.995345 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:11.995268    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:38:11.998798 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:38:11.998766    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef4fb952_b27b_44f8_a979_d4320c74bd2f.slice/crio-4dc4acb022ee95e09e1f6da52840557777a229e62aa5d5bb3fc735888f42efb2 WatchSource:0}: Error finding container 4dc4acb022ee95e09e1f6da52840557777a229e62aa5d5bb3fc735888f42efb2: Status 404 returned error can't find the container with id 4dc4acb022ee95e09e1f6da52840557777a229e62aa5d5bb3fc735888f42efb2
Sep 10 14:38:12.915239 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:12.915205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" event={"ID":"ef4fb952-b27b-44f8-a979-d4320c74bd2f","Type":"ContainerStarted","Data":"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"}
Sep 10 14:38:12.915627 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:12.915245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" event={"ID":"ef4fb952-b27b-44f8-a979-d4320c74bd2f","Type":"ContainerStarted","Data":"4dc4acb022ee95e09e1f6da52840557777a229e62aa5d5bb3fc735888f42efb2"}
Sep 10 14:38:12.931232 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:12.931187    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" podStartSLOduration=1.9311720559999999 podStartE2EDuration="1.931172056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:38:12.93010583 +0000 UTC m=+1604.891615349" watchObservedRunningTime="2026-09-10 14:38:12.931172056 +0000 UTC m=+1604.892681616"
Sep 10 14:38:21.719228 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:21.719105    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:21.719228 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:21.719201    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:21.731558 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:21.731537    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:21.970069 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:21.969977    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:38:43.161730 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.161683    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:38:43.162507 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.162453    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" podUID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" containerName="payload-pre-processing" containerID="cri-o://ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7" gracePeriod=30
Sep 10 14:38:43.415634 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.415569    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:38:43.570100 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.570065    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxknq\" (UniqueName: \"kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq\") pod \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") "
Sep 10 14:38:43.570246 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.570200    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume\") pod \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\" (UID: \"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e\") "
Sep 10 14:38:43.570579 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.570546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" (UID: "cb0b70a2-387b-42a3-8eb6-17bf4cadf88e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:38:43.572181 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.572160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq" (OuterVolumeSpecName: "kube-api-access-gxknq") pod "cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" (UID: "cb0b70a2-387b-42a3-8eb6-17bf4cadf88e"). InnerVolumeSpecName "kube-api-access-gxknq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:43.671942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.671868    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:43.671942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:43.671894    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxknq\" (UniqueName: \"kubernetes.io/projected/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e-kube-api-access-gxknq\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:38:44.075370 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.075334    2581 generic.go:356] "Generic (PLEG): container finished" podID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" containerID="ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7" exitCode=0
Sep 10 14:38:44.075605 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.075387    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"
Sep 10 14:38:44.075605 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.075419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" event={"ID":"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e","Type":"ContainerDied","Data":"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"}
Sep 10 14:38:44.075605 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.075495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t" event={"ID":"cb0b70a2-387b-42a3-8eb6-17bf4cadf88e","Type":"ContainerDied","Data":"cffbeb0ce62e364eecb734025a791d7b929d82bc698c76146e0518562091b369"}
Sep 10 14:38:44.075605 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.075518    2581 scope.go:117] "RemoveContainer" containerID="ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"
Sep 10 14:38:44.084745 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.084729    2581 scope.go:117] "RemoveContainer" containerID="ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"
Sep 10 14:38:44.085012 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:38:44.084993    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7\": container with ID starting with ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7 not found: ID does not exist" containerID="ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"
Sep 10 14:38:44.085061 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.085021    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7"} err="failed to get container status \"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7\": rpc error: code = NotFound desc = could not find container \"ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7\": container with ID starting with ff2a29e48c25c4668f459236645ce6b652638241868e2d0fa522d937fd3bd7e7 not found: ID does not exist"
Sep 10 14:38:44.097496 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.097472    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:38:44.100748 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.100727    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-54c040-5f575fdc7d-k4h5t"]
Sep 10 14:38:44.532228 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:38:44.532198    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" path="/var/lib/kubelet/pods/cb0b70a2-387b-42a3-8eb6-17bf4cadf88e/volumes"
Sep 10 14:39:01.466130 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:01.466095    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:39:16.485139 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.485092    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" podUID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" containerName="main" containerID="cri-o://8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3" gracePeriod=45
Sep 10 14:39:16.733635 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.733608    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:39:16.851572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851535    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.851572 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851572    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsd7r\" (UniqueName: \"kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.851852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.851852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.851852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.851852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851773    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs\") pod \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\" (UID: \"ef4fb952-b27b-44f8-a979-d4320c74bd2f\") "
Sep 10 14:39:16.852074 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home" (OuterVolumeSpecName: "home") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:16.852074 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851977    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:16.852074 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.851991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:16.852200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.852086    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-model-cache\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.852200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.852105    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tmp-dir\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.852200 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.852113    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-home\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.853634 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.853612    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r" (OuterVolumeSpecName: "kube-api-access-rsd7r") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "kube-api-access-rsd7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:39:16.853740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.853719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:39:16.853864 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.853844    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm" (OuterVolumeSpecName: "dshm") pod "ef4fb952-b27b-44f8-a979-d4320c74bd2f" (UID: "ef4fb952-b27b-44f8-a979-d4320c74bd2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:16.952487 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.952461    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef4fb952-b27b-44f8-a979-d4320c74bd2f-tls-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.952487 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.952485    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef4fb952-b27b-44f8-a979-d4320c74bd2f-dshm\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.952619 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:16.952499    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsd7r\" (UniqueName: \"kubernetes.io/projected/ef4fb952-b27b-44f8-a979-d4320c74bd2f-kube-api-access-rsd7r\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:39:17.249024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.248936    2581 generic.go:356] "Generic (PLEG): container finished" podID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" containerID="8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3" exitCode=0
Sep 10 14:39:17.249024 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.248993    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"
Sep 10 14:39:17.249258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.249018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" event={"ID":"ef4fb952-b27b-44f8-a979-d4320c74bd2f","Type":"ContainerDied","Data":"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"}
Sep 10 14:39:17.249258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.249058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4" event={"ID":"ef4fb952-b27b-44f8-a979-d4320c74bd2f","Type":"ContainerDied","Data":"4dc4acb022ee95e09e1f6da52840557777a229e62aa5d5bb3fc735888f42efb2"}
Sep 10 14:39:17.249258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.249075    2581 scope.go:117] "RemoveContainer" containerID="8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"
Sep 10 14:39:17.257761 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.257744    2581 scope.go:117] "RemoveContainer" containerID="8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"
Sep 10 14:39:17.257997 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:39:17.257978    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3\": container with ID starting with 8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3 not found: ID does not exist" containerID="8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"
Sep 10 14:39:17.258061 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.258009    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3"} err="failed to get container status \"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3\": rpc error: code = NotFound desc = could not find container \"8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3\": container with ID starting with 8c4be2abae04e817e4b5a4306b4d633605d091bd5751a865c7d8ad459791b0b3 not found: ID does not exist"
Sep 10 14:39:17.272057 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.272033    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:39:17.278910 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:17.278889    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/rate-test-model-9dcc39-kserve-756f748847-vxfk4"]
Sep 10 14:39:18.532158 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:18.532113    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" path="/var/lib/kubelet/pods/ef4fb952-b27b-44f8-a979-d4320c74bd2f/volumes"
Sep 10 14:39:28.694074 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:28.694042    2581 scope.go:117] "RemoveContainer" containerID="3bbb66b51d323bb71ec6c79f30382a9e18934be0fe4681ea59a5bd4195ec3702"
Sep 10 14:39:36.618125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618089    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618478    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" containerName="payload-pre-processing"
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618489    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" containerName="payload-pre-processing"
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618507    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" containerName="main"
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618512    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" containerName="main"
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618571    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef4fb952-b27b-44f8-a979-d4320c74bd2f" containerName="main"
Sep 10 14:39:36.618596 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.618583    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb0b70a2-387b-42a3-8eb6-17bf4cadf88e" containerName="payload-pre-processing"
Sep 10 14:39:36.621602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.621587    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.625550 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.625529    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6b9e31-dockercfg-d8g4b\""
Sep 10 14:39:36.625550 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.625545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6b9e31\""
Sep 10 14:39:36.633115 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.633091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:39:36.730891 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.730856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.731051 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.730910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9d6\" (UniqueName: \"kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.831845 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.831810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.832037 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.831865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9d6\" (UniqueName: \"kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.832557 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.832530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.841547 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.841519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9d6\" (UniqueName: \"kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6\") pod \"payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:36.932428 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:36.932346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:37.205213 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:37.205187    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:39:37.206905 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:39:37.206874    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9fa42a_bffa_4c07_a88a_cc459603ccf2.slice/crio-d01eef8d8278bc1c55ab04c6be29f066c78167fe93a95fd55cd8d92f358df040 WatchSource:0}: Error finding container d01eef8d8278bc1c55ab04c6be29f066c78167fe93a95fd55cd8d92f358df040: Status 404 returned error can't find the container with id d01eef8d8278bc1c55ab04c6be29f066c78167fe93a95fd55cd8d92f358df040
Sep 10 14:39:37.344646 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:37.344611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" event={"ID":"db9fa42a-bffa-4c07-a88a-cc459603ccf2","Type":"ContainerStarted","Data":"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"}
Sep 10 14:39:37.344825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:37.344661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" event={"ID":"db9fa42a-bffa-4c07-a88a-cc459603ccf2","Type":"ContainerStarted","Data":"d01eef8d8278bc1c55ab04c6be29f066c78167fe93a95fd55cd8d92f358df040"}
Sep 10 14:39:37.344825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:37.344740    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:39:37.363162 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:37.363114    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" podStartSLOduration=1.363099214 podStartE2EDuration="1.363099214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:37.361801378 +0000 UTC m=+1689.323310896" watchObservedRunningTime="2026-09-10 14:39:37.363099214 +0000 UTC m=+1689.324608772"
Sep 10 14:39:48.351887 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:39:48.351854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:40:50.662150 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.662112    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:40:50.665924 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.665892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.668555 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.668531    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-652186-serving-cert\""
Sep 10 14:40:50.679857 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.679830    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:40:50.793953 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.793914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.794131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.793988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.794131 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.794011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwvg8\" (UniqueName: \"kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.794204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.794133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.895286 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.895242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.895464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.895394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.895464 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.895415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwvg8\" (UniqueName: \"kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.895543 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.895473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.895579 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:40:50.895544    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-serving-cert: secret "maas-api-e2e-ipp-cleanup-652186-serving-cert" not found
Sep 10 14:40:50.895654 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:40:50.895640    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls podName:698b6bc3-c2a9-4356-a974-e9aa10138319 nodeName:}" failed. No retries permitted until 2026-09-10 14:40:51.395617803 +0000 UTC m=+1763.357127304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" (UID: "698b6bc3-c2a9-4356-a974-e9aa10138319") : secret "maas-api-e2e-ipp-cleanup-652186-serving-cert" not found
Sep 10 14:40:50.896001 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.895984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.897957 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.897938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:50.904178 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:50.904157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwvg8\" (UniqueName: \"kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:51.328217 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.328171    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:40:51.332081 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.332059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.334956 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.334934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-652186-dockercfg-6dcps\""
Sep 10 14:40:51.335097 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.335076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-652186\""
Sep 10 14:40:51.345219 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.345192    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:40:51.400468 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.400431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bznr\" (UniqueName: \"kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.400683 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.400477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.400683 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.400550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:51.402980 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.402960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:51.501978 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.501941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bznr\" (UniqueName: \"kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.502156 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.501994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.502777 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.502758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.513984 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.513963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bznr\" (UniqueName: \"kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr\") pod \"payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.589478 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.589391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:51.645027 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.644996    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:51.866236 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.866206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:40:51.867947 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:40:51.867913    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698b6bc3_c2a9_4356_a974_e9aa10138319.slice/crio-6d5e7149c840fcdc10f718b4016de5806afb05e70de99e9b94cecdbbfc519474 WatchSource:0}: Error finding container 6d5e7149c840fcdc10f718b4016de5806afb05e70de99e9b94cecdbbfc519474: Status 404 returned error can't find the container with id 6d5e7149c840fcdc10f718b4016de5806afb05e70de99e9b94cecdbbfc519474
Sep 10 14:40:51.930387 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:51.930361    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:40:51.932454 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:40:51.932427    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d9c0c9_0ca5_400e_814a_5ccf92505569.slice/crio-f0e50180e2d7b91621994d1cb9f8ec67de510e17af885bae3928dc9b18697940 WatchSource:0}: Error finding container f0e50180e2d7b91621994d1cb9f8ec67de510e17af885bae3928dc9b18697940: Status 404 returned error can't find the container with id f0e50180e2d7b91621994d1cb9f8ec67de510e17af885bae3928dc9b18697940
Sep 10 14:40:52.719750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.719640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" event={"ID":"698b6bc3-c2a9-4356-a974-e9aa10138319","Type":"ContainerStarted","Data":"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"}
Sep 10 14:40:52.719750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.719727    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:40:52.720007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.719756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" event={"ID":"698b6bc3-c2a9-4356-a974-e9aa10138319","Type":"ContainerStarted","Data":"6d5e7149c840fcdc10f718b4016de5806afb05e70de99e9b94cecdbbfc519474"}
Sep 10 14:40:52.721308 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.721282    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" event={"ID":"29d9c0c9-0ca5-400e-814a-5ccf92505569","Type":"ContainerStarted","Data":"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"}
Sep 10 14:40:52.721437 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.721330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" event={"ID":"29d9c0c9-0ca5-400e-814a-5ccf92505569","Type":"ContainerStarted","Data":"f0e50180e2d7b91621994d1cb9f8ec67de510e17af885bae3928dc9b18697940"}
Sep 10 14:40:52.721437 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.721368    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:40:52.746674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.746622    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" podStartSLOduration=2.2865007090000002 podStartE2EDuration="2.746607231s" totalImagesPullingTime="460.106522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:52.745580961 +0000 UTC m=+1764.707090483" watchObservedRunningTime="2026-09-10 14:40:52.746607231 +0000 UTC m=+1764.708116749"
Sep 10 14:40:52.804416 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:52.804360    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" podStartSLOduration=1.804342497 podStartE2EDuration="1.804342497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:52.803620739 +0000 UTC m=+1764.765130241" watchObservedRunningTime="2026-09-10 14:40:52.804342497 +0000 UTC m=+1764.765852016"
Sep 10 14:40:58.735364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:40:58.735334    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:41:03.732750 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:03.732717    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:41:21.364866 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.364780    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:41:21.365436 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.365030    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" podUID="698b6bc3-c2a9-4356-a974-e9aa10138319" containerName="maas-api" containerID="cri-o://a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605" gracePeriod=30
Sep 10 14:41:21.416518 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.416481    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:41:21.416789 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.416758    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" podUID="29d9c0c9-0ca5-400e-814a-5ccf92505569" containerName="payload-processing" containerID="cri-o://1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05" gracePeriod=30
Sep 10 14:41:21.677746 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.677722    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:41:21.698687 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.698658    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:41:21.786945 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.786911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwvg8\" (UniqueName: \"kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8\") pod \"698b6bc3-c2a9-4356-a974-e9aa10138319\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") "
Sep 10 14:41:21.787125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.786970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca\") pod \"698b6bc3-c2a9-4356-a974-e9aa10138319\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") "
Sep 10 14:41:21.787125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787007    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs\") pod \"698b6bc3-c2a9-4356-a974-e9aa10138319\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") "
Sep 10 14:41:21.787125 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") pod \"698b6bc3-c2a9-4356-a974-e9aa10138319\" (UID: \"698b6bc3-c2a9-4356-a974-e9aa10138319\") "
Sep 10 14:41:21.787289 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787128    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bznr\" (UniqueName: \"kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr\") pod \"29d9c0c9-0ca5-400e-814a-5ccf92505569\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") "
Sep 10 14:41:21.787289 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787160    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume\") pod \"29d9c0c9-0ca5-400e-814a-5ccf92505569\" (UID: \"29d9c0c9-0ca5-400e-814a-5ccf92505569\") "
Sep 10 14:41:21.787410 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "698b6bc3-c2a9-4356-a974-e9aa10138319" (UID: "698b6bc3-c2a9-4356-a974-e9aa10138319"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:21.787502 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787477    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/698b6bc3-c2a9-4356-a974-e9aa10138319-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.787585 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.787563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "29d9c0c9-0ca5-400e-814a-5ccf92505569" (UID: "29d9c0c9-0ca5-400e-814a-5ccf92505569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:21.789188 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.789161    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "698b6bc3-c2a9-4356-a974-e9aa10138319" (UID: "698b6bc3-c2a9-4356-a974-e9aa10138319"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:21.789293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.789192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "698b6bc3-c2a9-4356-a974-e9aa10138319" (UID: "698b6bc3-c2a9-4356-a974-e9aa10138319"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:21.789565 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.789546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr" (OuterVolumeSpecName: "kube-api-access-4bznr") pod "29d9c0c9-0ca5-400e-814a-5ccf92505569" (UID: "29d9c0c9-0ca5-400e-814a-5ccf92505569"). InnerVolumeSpecName "kube-api-access-4bznr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:21.789622 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.789557    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8" (OuterVolumeSpecName: "kube-api-access-gwvg8") pod "698b6bc3-c2a9-4356-a974-e9aa10138319" (UID: "698b6bc3-c2a9-4356-a974-e9aa10138319"). InnerVolumeSpecName "kube-api-access-gwvg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:21.884216 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.884120    2581 generic.go:356] "Generic (PLEG): container finished" podID="29d9c0c9-0ca5-400e-814a-5ccf92505569" containerID="1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05" exitCode=0
Sep 10 14:41:21.884216 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.884166    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"
Sep 10 14:41:21.884216 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.884199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" event={"ID":"29d9c0c9-0ca5-400e-814a-5ccf92505569","Type":"ContainerDied","Data":"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"}
Sep 10 14:41:21.884462 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.884248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh" event={"ID":"29d9c0c9-0ca5-400e-814a-5ccf92505569","Type":"ContainerDied","Data":"f0e50180e2d7b91621994d1cb9f8ec67de510e17af885bae3928dc9b18697940"}
Sep 10 14:41:21.884462 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.884274    2581 scope.go:117] "RemoveContainer" containerID="1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"
Sep 10 14:41:21.885611 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.885585    2581 generic.go:356] "Generic (PLEG): container finished" podID="698b6bc3-c2a9-4356-a974-e9aa10138319" containerID="a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605" exitCode=0
Sep 10 14:41:21.885740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.885624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"
Sep 10 14:41:21.885740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.885635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" event={"ID":"698b6bc3-c2a9-4356-a974-e9aa10138319","Type":"ContainerDied","Data":"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"}
Sep 10 14:41:21.885740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.885670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh" event={"ID":"698b6bc3-c2a9-4356-a974-e9aa10138319","Type":"ContainerDied","Data":"6d5e7149c840fcdc10f718b4016de5806afb05e70de99e9b94cecdbbfc519474"}
Sep 10 14:41:21.888141 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.888116    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bznr\" (UniqueName: \"kubernetes.io/projected/29d9c0c9-0ca5-400e-814a-5ccf92505569-kube-api-access-4bznr\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.888266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.888144    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29d9c0c9-0ca5-400e-814a-5ccf92505569-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.888266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.888160    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwvg8\" (UniqueName: \"kubernetes.io/projected/698b6bc3-c2a9-4356-a974-e9aa10138319-kube-api-access-gwvg8\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.888266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.888174    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.888266 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.888187    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/698b6bc3-c2a9-4356-a974-e9aa10138319-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:41:21.896332 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.896313    2581 scope.go:117] "RemoveContainer" containerID="1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"
Sep 10 14:41:21.896629 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:41:21.896609    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05\": container with ID starting with 1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05 not found: ID does not exist" containerID="1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"
Sep 10 14:41:21.896738 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.896636    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05"} err="failed to get container status \"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05\": rpc error: code = NotFound desc = could not find container \"1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05\": container with ID starting with 1d59e8df308042d05cc8183a8cf6e7aab1300f91cf1321368a02c5585bb03b05 not found: ID does not exist"
Sep 10 14:41:21.896738 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.896653    2581 scope.go:117] "RemoveContainer" containerID="a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"
Sep 10 14:41:21.905361 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.905340    2581 scope.go:117] "RemoveContainer" containerID="a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"
Sep 10 14:41:21.905592 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:41:21.905573    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605\": container with ID starting with a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605 not found: ID does not exist" containerID="a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"
Sep 10 14:41:21.905641 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.905600    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605"} err="failed to get container status \"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605\": rpc error: code = NotFound desc = could not find container \"a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605\": container with ID starting with a43487cbb71fa25d8722359602468bdd6451d5f958e959edda8888eb22a83605 not found: ID does not exist"
Sep 10 14:41:21.910942 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.910916    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:41:21.914667 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.914647    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-652186-6f8455664c-5s6kh"]
Sep 10 14:41:21.925388 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.925366    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:41:21.930852 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:21.930832    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-652186-77ffdfdd4c-ngjrh"]
Sep 10 14:41:22.532491 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:22.532459    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29d9c0c9-0ca5-400e-814a-5ccf92505569" path="/var/lib/kubelet/pods/29d9c0c9-0ca5-400e-814a-5ccf92505569/volumes"
Sep 10 14:41:22.532906 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:22.532891    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698b6bc3-c2a9-4356-a974-e9aa10138319" path="/var/lib/kubelet/pods/698b6bc3-c2a9-4356-a974-e9aa10138319/volumes"
Sep 10 14:41:28.580264 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:28.580232    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:41:28.587898 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:41:28.587869    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:42:51.717934 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:51.717852    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:42:51.718439 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:51.718090    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" podUID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" containerName="payload-pre-processing" containerID="cri-o://3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0" gracePeriod=30
Sep 10 14:42:51.967760 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:51.967731    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:42:52.022422 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.022376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume\") pod \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") "
Sep 10 14:42:52.022591 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.022489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq9d6\" (UniqueName: \"kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6\") pod \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\" (UID: \"db9fa42a-bffa-4c07-a88a-cc459603ccf2\") "
Sep 10 14:42:52.022768 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.022742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db9fa42a-bffa-4c07-a88a-cc459603ccf2" (UID: "db9fa42a-bffa-4c07-a88a-cc459603ccf2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:52.024468 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.024447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6" (OuterVolumeSpecName: "kube-api-access-zq9d6") pod "db9fa42a-bffa-4c07-a88a-cc459603ccf2" (UID: "db9fa42a-bffa-4c07-a88a-cc459603ccf2"). InnerVolumeSpecName "kube-api-access-zq9d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:52.123631 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.123574    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq9d6\" (UniqueName: \"kubernetes.io/projected/db9fa42a-bffa-4c07-a88a-cc459603ccf2-kube-api-access-zq9d6\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.123631 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.123625    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db9fa42a-bffa-4c07-a88a-cc459603ccf2-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.325909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.325810    2581 generic.go:356] "Generic (PLEG): container finished" podID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" containerID="3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0" exitCode=0
Sep 10 14:42:52.325909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.325852    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"
Sep 10 14:42:52.325909 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.325888    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" event={"ID":"db9fa42a-bffa-4c07-a88a-cc459603ccf2","Type":"ContainerDied","Data":"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"}
Sep 10 14:42:52.326184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.325941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4" event={"ID":"db9fa42a-bffa-4c07-a88a-cc459603ccf2","Type":"ContainerDied","Data":"d01eef8d8278bc1c55ab04c6be29f066c78167fe93a95fd55cd8d92f358df040"}
Sep 10 14:42:52.326184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.325966    2581 scope.go:117] "RemoveContainer" containerID="3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"
Sep 10 14:42:52.335326 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.335304    2581 scope.go:117] "RemoveContainer" containerID="3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"
Sep 10 14:42:52.335593 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:42:52.335571    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0\": container with ID starting with 3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0 not found: ID does not exist" containerID="3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"
Sep 10 14:42:52.335669 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.335607    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0"} err="failed to get container status \"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0\": rpc error: code = NotFound desc = could not find container \"3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0\": container with ID starting with 3a7b38ffe38f5202d3d7e1590910dc2cf5bce150a119807f99c73737574c50a0 not found: ID does not exist"
Sep 10 14:42:52.347055 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.347031    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:42:52.350944 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.350918    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6b9e31-54c6849ccd-72fx4"]
Sep 10 14:42:52.531949 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:42:52.531914    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" path="/var/lib/kubelet/pods/db9fa42a-bffa-4c07-a88a-cc459603ccf2/volumes"
Sep 10 14:43:09.096821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.096784    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:43:09.097394 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.097044    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" podUID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" containerName="maas-api" containerID="cri-o://455f065fea3a7a339668bd12a61156aa6feacdf857023651b586304a19421d4a" gracePeriod=30
Sep 10 14:43:09.151838 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.151798    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:43:09.152121 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.152067    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" podUID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" containerName="payload-processing" containerID="cri-o://de9db34f6ada58e9dd174f3110132d531f673d31cde4323d21a3b81325c96149" gracePeriod=30
Sep 10 14:43:09.410544 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.410514    2581 generic.go:356] "Generic (PLEG): container finished" podID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" containerID="455f065fea3a7a339668bd12a61156aa6feacdf857023651b586304a19421d4a" exitCode=0
Sep 10 14:43:09.410684 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.410593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" event={"ID":"1e3c24de-f263-4f2d-a265-cb2ea4a61dff","Type":"ContainerDied","Data":"455f065fea3a7a339668bd12a61156aa6feacdf857023651b586304a19421d4a"}
Sep 10 14:43:09.410684 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.410649    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt" event={"ID":"1e3c24de-f263-4f2d-a265-cb2ea4a61dff","Type":"ContainerDied","Data":"feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab"}
Sep 10 14:43:09.410684 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.410669    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feff6852a75d63d7c5189d82189d00c67862df98401558e98c5f6dd753bfafab"
Sep 10 14:43:09.411919 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.411898    2581 generic.go:356] "Generic (PLEG): container finished" podID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" containerID="de9db34f6ada58e9dd174f3110132d531f673d31cde4323d21a3b81325c96149" exitCode=0
Sep 10 14:43:09.412018 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.411936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" event={"ID":"09ba97a0-e0e7-47e5-831b-2b6c04ab530a","Type":"ContainerDied","Data":"de9db34f6ada58e9dd174f3110132d531f673d31cde4323d21a3b81325c96149"}
Sep 10 14:43:09.425490 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.425470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:43:09.430876 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.430857    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:43:09.594079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume\") pod \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") "
Sep 10 14:43:09.594079 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594089    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24ksp\" (UniqueName: \"kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp\") pod \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\" (UID: \"09ba97a0-e0e7-47e5-831b-2b6c04ab530a\") "
Sep 10 14:43:09.594364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594113    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs\") pod \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") "
Sep 10 14:43:09.594364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594171    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca\") pod \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") "
Sep 10 14:43:09.594364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594188    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg2f9\" (UniqueName: \"kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9\") pod \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") "
Sep 10 14:43:09.594364 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594245    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") pod \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\" (UID: \"1e3c24de-f263-4f2d-a265-cb2ea4a61dff\") "
Sep 10 14:43:09.594576 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09ba97a0-e0e7-47e5-831b-2b6c04ab530a" (UID: "09ba97a0-e0e7-47e5-831b-2b6c04ab530a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:09.594576 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.594528    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e3c24de-f263-4f2d-a265-cb2ea4a61dff" (UID: "1e3c24de-f263-4f2d-a265-cb2ea4a61dff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:09.596397 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.596369    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp" (OuterVolumeSpecName: "kube-api-access-24ksp") pod "09ba97a0-e0e7-47e5-831b-2b6c04ab530a" (UID: "09ba97a0-e0e7-47e5-831b-2b6c04ab530a"). InnerVolumeSpecName "kube-api-access-24ksp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:09.596397 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.596384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1e3c24de-f263-4f2d-a265-cb2ea4a61dff" (UID: "1e3c24de-f263-4f2d-a265-cb2ea4a61dff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:09.596540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.596407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e3c24de-f263-4f2d-a265-cb2ea4a61dff" (UID: "1e3c24de-f263-4f2d-a265-cb2ea4a61dff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:09.596540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.596432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9" (OuterVolumeSpecName: "kube-api-access-hg2f9") pod "1e3c24de-f263-4f2d-a265-cb2ea4a61dff" (UID: "1e3c24de-f263-4f2d-a265-cb2ea4a61dff"). InnerVolumeSpecName "kube-api-access-hg2f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695402    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695439    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24ksp\" (UniqueName: \"kubernetes.io/projected/09ba97a0-e0e7-47e5-831b-2b6c04ab530a-kube-api-access-24ksp\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695455    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695470    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695485    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg2f9\" (UniqueName: \"kubernetes.io/projected/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-kube-api-access-hg2f9\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:09.695503 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:09.695500    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3c24de-f263-4f2d-a265-cb2ea4a61dff-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:10.418166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.418085    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"
Sep 10 14:43:10.418166 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.418120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n" event={"ID":"09ba97a0-e0e7-47e5-831b-2b6c04ab530a","Type":"ContainerDied","Data":"873443c0978eeac2b310559d866baccb98a2251ffffc3108ef4c2f07560bb046"}
Sep 10 14:43:10.418674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.418177    2581 scope.go:117] "RemoveContainer" containerID="de9db34f6ada58e9dd174f3110132d531f673d31cde4323d21a3b81325c96149"
Sep 10 14:43:10.418674 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.418301    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"
Sep 10 14:43:10.443434 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.443408    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:43:10.447969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.447948    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-586826-54c7979f4b-66qxt"]
Sep 10 14:43:10.459258 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.459233    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:43:10.463427 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.463406    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-586826-6ddf847cc-67g4n"]
Sep 10 14:43:10.531764 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.531731    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" path="/var/lib/kubelet/pods/09ba97a0-e0e7-47e5-831b-2b6c04ab530a/volumes"
Sep 10 14:43:10.532159 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:10.532146    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" path="/var/lib/kubelet/pods/1e3c24de-f263-4f2d-a265-cb2ea4a61dff/volumes"
Sep 10 14:43:26.357372 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.357335    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:43:26.357881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.357584    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" podUID="6acb7831-46a3-44db-af96-34f39036666f" containerName="maas-api" containerID="cri-o://a1d3174b9897233133566e0d10e5eff0854f393028d02446847109bb52888247" gracePeriod=30
Sep 10 14:43:26.416693 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.416657    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:43:26.416961 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.416929    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" podUID="9bad7d32-e805-4aef-a751-b696ca738b68" containerName="payload-processing" containerID="cri-o://0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8" gracePeriod=30
Sep 10 14:43:26.501387 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.501352    2581 generic.go:356] "Generic (PLEG): container finished" podID="6acb7831-46a3-44db-af96-34f39036666f" containerID="a1d3174b9897233133566e0d10e5eff0854f393028d02446847109bb52888247" exitCode=0
Sep 10 14:43:26.501548 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.501391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" event={"ID":"6acb7831-46a3-44db-af96-34f39036666f","Type":"ContainerDied","Data":"a1d3174b9897233133566e0d10e5eff0854f393028d02446847109bb52888247"}
Sep 10 14:43:26.701400 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.701374    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:43:26.704091 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.704069    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:43:26.750346 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume\") pod \"9bad7d32-e805-4aef-a751-b696ca738b68\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") "
Sep 10 14:43:26.750542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f2fg\" (UniqueName: \"kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg\") pod \"9bad7d32-e805-4aef-a751-b696ca738b68\" (UID: \"9bad7d32-e805-4aef-a751-b696ca738b68\") "
Sep 10 14:43:26.750542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750430    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca\") pod \"6acb7831-46a3-44db-af96-34f39036666f\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") "
Sep 10 14:43:26.750542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs\") pod \"6acb7831-46a3-44db-af96-34f39036666f\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") "
Sep 10 14:43:26.750542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls\") pod \"6acb7831-46a3-44db-af96-34f39036666f\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") "
Sep 10 14:43:26.750825 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750560    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfgbb\" (UniqueName: \"kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb\") pod \"6acb7831-46a3-44db-af96-34f39036666f\" (UID: \"6acb7831-46a3-44db-af96-34f39036666f\") "
Sep 10 14:43:26.750908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750875    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bad7d32-e805-4aef-a751-b696ca738b68" (UID: "9bad7d32-e805-4aef-a751-b696ca738b68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:26.750970 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.750894    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6acb7831-46a3-44db-af96-34f39036666f" (UID: "6acb7831-46a3-44db-af96-34f39036666f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:26.753014 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.752981    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6acb7831-46a3-44db-af96-34f39036666f" (UID: "6acb7831-46a3-44db-af96-34f39036666f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:26.753169 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.753148    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg" (OuterVolumeSpecName: "kube-api-access-8f2fg") pod "9bad7d32-e805-4aef-a751-b696ca738b68" (UID: "9bad7d32-e805-4aef-a751-b696ca738b68"). InnerVolumeSpecName "kube-api-access-8f2fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:26.753450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.753426    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb" (OuterVolumeSpecName: "kube-api-access-tfgbb") pod "6acb7831-46a3-44db-af96-34f39036666f" (UID: "6acb7831-46a3-44db-af96-34f39036666f"). InnerVolumeSpecName "kube-api-access-tfgbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:26.753526 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.753499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6acb7831-46a3-44db-af96-34f39036666f" (UID: "6acb7831-46a3-44db-af96-34f39036666f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:26.851969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.851931    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bad7d32-e805-4aef-a751-b696ca738b68-plugins-config-volume\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:26.851969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.851961    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f2fg\" (UniqueName: \"kubernetes.io/projected/9bad7d32-e805-4aef-a751-b696ca738b68-kube-api-access-8f2fg\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:26.851969 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.851974    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acb7831-46a3-44db-af96-34f39036666f-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:26.852217 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.851986    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:26.852217 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.851998    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6acb7831-46a3-44db-af96-34f39036666f-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:26.852217 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:26.852009    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfgbb\" (UniqueName: \"kubernetes.io/projected/6acb7831-46a3-44db-af96-34f39036666f-kube-api-access-tfgbb\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:27.507184 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.507146    2581 generic.go:356] "Generic (PLEG): container finished" podID="9bad7d32-e805-4aef-a751-b696ca738b68" containerID="0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8" exitCode=0
Sep 10 14:43:27.507649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.507194    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"
Sep 10 14:43:27.507649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.507225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" event={"ID":"9bad7d32-e805-4aef-a751-b696ca738b68","Type":"ContainerDied","Data":"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"}
Sep 10 14:43:27.507649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.507285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g" event={"ID":"9bad7d32-e805-4aef-a751-b696ca738b68","Type":"ContainerDied","Data":"58ca5b1a4ab676037b58740fd82feff7e32e698cc45cbc76faff89bf950c2dc2"}
Sep 10 14:43:27.507649 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.507317    2581 scope.go:117] "RemoveContainer" containerID="0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"
Sep 10 14:43:27.508777 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.508758    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"
Sep 10 14:43:27.508859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.508759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr" event={"ID":"6acb7831-46a3-44db-af96-34f39036666f","Type":"ContainerDied","Data":"a892d4caed6ee97501bbd9c74c6667da5eae3ddcd369fd027f325ae50f0a5027"}
Sep 10 14:43:27.516044 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.516026    2581 scope.go:117] "RemoveContainer" containerID="0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"
Sep 10 14:43:27.516279 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:43:27.516258    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8\": container with ID starting with 0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8 not found: ID does not exist" containerID="0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"
Sep 10 14:43:27.516327 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.516290    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8"} err="failed to get container status \"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8\": rpc error: code = NotFound desc = could not find container \"0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8\": container with ID starting with 0ab9f09dcf5218d2b6c11dd660b9dc1bf588f81b866e312dfd6ca1633b0e5cd8 not found: ID does not exist"
Sep 10 14:43:27.516327 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.516311    2581 scope.go:117] "RemoveContainer" containerID="a1d3174b9897233133566e0d10e5eff0854f393028d02446847109bb52888247"
Sep 10 14:43:27.541656 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.541632    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:43:27.545630 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.545608    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9dcc39-8649844f55-ggz2g"]
Sep 10 14:43:27.561331 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.561304    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:43:27.563916 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:27.563894    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9dcc39-9848b9557-nbddr"]
Sep 10 14:43:28.532018 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:28.531979    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6acb7831-46a3-44db-af96-34f39036666f" path="/var/lib/kubelet/pods/6acb7831-46a3-44db-af96-34f39036666f/volumes"
Sep 10 14:43:28.532532 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:28.532510    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bad7d32-e805-4aef-a751-b696ca738b68" path="/var/lib/kubelet/pods/9bad7d32-e805-4aef-a751-b696ca738b68/volumes"
Sep 10 14:43:28.748897 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:28.748870    2581 scope.go:117] "RemoveContainer" containerID="455f065fea3a7a339668bd12a61156aa6feacdf857023651b586304a19421d4a"
Sep 10 14:43:35.581222 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.581188    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:43:35.581754 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.581435    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-764f45744-zsgfj" podUID="170def59-59d9-4e5f-aac7-17ca255c293f" containerName="manager" containerID="cri-o://1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c" gracePeriod=30
Sep 10 14:43:35.828822 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.828798    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:43:35.931796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.931521    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g76js\" (UniqueName: \"kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js\") pod \"170def59-59d9-4e5f-aac7-17ca255c293f\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") "
Sep 10 14:43:35.931996 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.931976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") pod \"170def59-59d9-4e5f-aac7-17ca255c293f\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") "
Sep 10 14:43:35.932072 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.932060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs\") pod \"170def59-59d9-4e5f-aac7-17ca255c293f\" (UID: \"170def59-59d9-4e5f-aac7-17ca255c293f\") "
Sep 10 14:43:35.934542 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.934507    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "170def59-59d9-4e5f-aac7-17ca255c293f" (UID: "170def59-59d9-4e5f-aac7-17ca255c293f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:35.940348 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.940311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js" (OuterVolumeSpecName: "kube-api-access-g76js") pod "170def59-59d9-4e5f-aac7-17ca255c293f" (UID: "170def59-59d9-4e5f-aac7-17ca255c293f"). InnerVolumeSpecName "kube-api-access-g76js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:35.940963 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:35.940842    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "170def59-59d9-4e5f-aac7-17ca255c293f" (UID: "170def59-59d9-4e5f-aac7-17ca255c293f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:36.033981 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.033945    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-webhook-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:36.033981 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.033973    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/170def59-59d9-4e5f-aac7-17ca255c293f-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:36.033981 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.033982    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g76js\" (UniqueName: \"kubernetes.io/projected/170def59-59d9-4e5f-aac7-17ca255c293f-kube-api-access-g76js\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:43:36.553747 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.553715    2581 generic.go:356] "Generic (PLEG): container finished" podID="170def59-59d9-4e5f-aac7-17ca255c293f" containerID="1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c" exitCode=0
Sep 10 14:43:36.553893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.553772    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-zsgfj"
Sep 10 14:43:36.553893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.553794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-zsgfj" event={"ID":"170def59-59d9-4e5f-aac7-17ca255c293f","Type":"ContainerDied","Data":"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"}
Sep 10 14:43:36.553893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.553846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-zsgfj" event={"ID":"170def59-59d9-4e5f-aac7-17ca255c293f","Type":"ContainerDied","Data":"7b7a7bbe9199961e8937d1a0b3740cbce6b8566f6d9747997cce80c71815240a"}
Sep 10 14:43:36.553893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.553869    2581 scope.go:117] "RemoveContainer" containerID="1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"
Sep 10 14:43:36.563090 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.563070    2581 scope.go:117] "RemoveContainer" containerID="1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"
Sep 10 14:43:36.563353 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:43:36.563335    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c\": container with ID starting with 1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c not found: ID does not exist" containerID="1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"
Sep 10 14:43:36.563426 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.563361    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c"} err="failed to get container status \"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c\": rpc error: code = NotFound desc = could not find container \"1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c\": container with ID starting with 1f1a718c25128ec5a36987715ff41539433bd862e4022dc20b41c15fe519228c not found: ID does not exist"
Sep 10 14:43:36.570457 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.570426    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:43:36.572594 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:36.572570    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-764f45744-zsgfj"]
Sep 10 14:43:38.531368 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:43:38.531333    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170def59-59d9-4e5f-aac7-17ca255c293f" path="/var/lib/kubelet/pods/170def59-59d9-4e5f-aac7-17ca255c293f/volumes"
Sep 10 14:44:37.356905 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.356869    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357206    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357218    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357229    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698b6bc3-c2a9-4356-a974-e9aa10138319" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357234    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="698b6bc3-c2a9-4356-a974-e9aa10138319" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357250    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d9c0c9-0ca5-400e-814a-5ccf92505569" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357256    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d9c0c9-0ca5-400e-814a-5ccf92505569" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357263    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6acb7831-46a3-44db-af96-34f39036666f" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357268    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6acb7831-46a3-44db-af96-34f39036666f" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357278    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bad7d32-e805-4aef-a751-b696ca738b68" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357283    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bad7d32-e805-4aef-a751-b696ca738b68" containerName="payload-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357291    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170def59-59d9-4e5f-aac7-17ca255c293f" containerName="manager"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357296    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="170def59-59d9-4e5f-aac7-17ca255c293f" containerName="manager"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357301    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357306    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" containerName="maas-api"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357312    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" containerName="payload-pre-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357316    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" containerName="payload-pre-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357365    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9fa42a-bffa-4c07-a88a-cc459603ccf2" containerName="payload-pre-processing"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357372    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="170def59-59d9-4e5f-aac7-17ca255c293f" containerName="manager"
Sep 10 14:44:37.357367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357378    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e3c24de-f263-4f2d-a265-cb2ea4a61dff" containerName="maas-api"
Sep 10 14:44:37.357992 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357384    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="698b6bc3-c2a9-4356-a974-e9aa10138319" containerName="maas-api"
Sep 10 14:44:37.357992 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357391    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ba97a0-e0e7-47e5-831b-2b6c04ab530a" containerName="payload-processing"
Sep 10 14:44:37.357992 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357399    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d9c0c9-0ca5-400e-814a-5ccf92505569" containerName="payload-processing"
Sep 10 14:44:37.357992 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357405    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bad7d32-e805-4aef-a751-b696ca738b68" containerName="payload-processing"
Sep 10 14:44:37.357992 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.357410    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6acb7831-46a3-44db-af96-34f39036666f" containerName="maas-api"
Sep 10 14:44:37.360136 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.360120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.363172 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.363151    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:44:37.363309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.363152    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:44:37.363309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.363211    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t8r42\""
Sep 10 14:44:37.363309 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.363226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:44:37.371594 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.371573    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:44:37.500372 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.500337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.500557 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.500389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.500557 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.500424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lz6q\" (UniqueName: \"kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.601796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.601758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.601982 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.601818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.602068 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.602043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lz6q\" (UniqueName: \"kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.604267 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.604245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.604396 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.604274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.614450 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.614388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lz6q\" (UniqueName: \"kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q\") pod \"maas-controller-764f45744-tlz29\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") " pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.671537 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.671512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:37.942918 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.942891    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:44:37.945368 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:44:37.945343    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc8335c_3f3a_414d_a00a_7821ed40fdb8.slice/crio-b6ff7bca0bf1c87935314d027b110494aec28956e9df886b5f68b468e00c423a WatchSource:0}: Error finding container b6ff7bca0bf1c87935314d027b110494aec28956e9df886b5f68b468e00c423a: Status 404 returned error can't find the container with id b6ff7bca0bf1c87935314d027b110494aec28956e9df886b5f68b468e00c423a
Sep 10 14:44:37.946752 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:37.946736    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:44:38.838693 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:38.838650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-tlz29" event={"ID":"cfc8335c-3f3a-414d-a00a-7821ed40fdb8","Type":"ContainerStarted","Data":"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"}
Sep 10 14:44:38.839157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:38.838729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-tlz29" event={"ID":"cfc8335c-3f3a-414d-a00a-7821ed40fdb8","Type":"ContainerStarted","Data":"b6ff7bca0bf1c87935314d027b110494aec28956e9df886b5f68b468e00c423a"}
Sep 10 14:44:38.839157 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:38.838834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:38.858758 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:38.858711    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-764f45744-tlz29" podStartSLOduration=1.325389553 podStartE2EDuration="1.85868385s" totalImagesPullingTime="533.294297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:38.856254927 +0000 UTC m=+1990.817764446" watchObservedRunningTime="2026-09-10 14:44:38.85868385 +0000 UTC m=+1990.820193368"
Sep 10 14:44:49.845850 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:49.845819    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:59.143178 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.143137    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:44:59.143662 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.143350    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-764f45744-tlz29" podUID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" containerName="manager" containerID="cri-o://5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961" gracePeriod=30
Sep 10 14:44:59.388178 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.388147    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:59.492007 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.491917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lz6q\" (UniqueName: \"kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q\") pod \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") "
Sep 10 14:44:59.492172 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.492014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert\") pod \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") "
Sep 10 14:44:59.492172 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.492057    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs\") pod \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\" (UID: \"cfc8335c-3f3a-414d-a00a-7821ed40fdb8\") "
Sep 10 14:44:59.494142 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.494107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q" (OuterVolumeSpecName: "kube-api-access-8lz6q") pod "cfc8335c-3f3a-414d-a00a-7821ed40fdb8" (UID: "cfc8335c-3f3a-414d-a00a-7821ed40fdb8"). InnerVolumeSpecName "kube-api-access-8lz6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:44:59.494283 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.494176    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfc8335c-3f3a-414d-a00a-7821ed40fdb8" (UID: "cfc8335c-3f3a-414d-a00a-7821ed40fdb8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:59.494283 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.494256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cfc8335c-3f3a-414d-a00a-7821ed40fdb8" (UID: "cfc8335c-3f3a-414d-a00a-7821ed40fdb8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:59.593461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.593428    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-webhook-cert\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:44:59.593461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.593453    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:44:59.593461 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.593464    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lz6q\" (UniqueName: \"kubernetes.io/projected/cfc8335c-3f3a-414d-a00a-7821ed40fdb8-kube-api-access-8lz6q\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:44:59.935855 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.935820    2581 generic.go:356] "Generic (PLEG): container finished" podID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" containerID="5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961" exitCode=0
Sep 10 14:44:59.936031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.935870    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-tlz29"
Sep 10 14:44:59.936031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.935887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-tlz29" event={"ID":"cfc8335c-3f3a-414d-a00a-7821ed40fdb8","Type":"ContainerDied","Data":"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"}
Sep 10 14:44:59.936031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.935927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-tlz29" event={"ID":"cfc8335c-3f3a-414d-a00a-7821ed40fdb8","Type":"ContainerDied","Data":"b6ff7bca0bf1c87935314d027b110494aec28956e9df886b5f68b468e00c423a"}
Sep 10 14:44:59.936031 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.935951    2581 scope.go:117] "RemoveContainer" containerID="5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"
Sep 10 14:44:59.944831 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.944810    2581 scope.go:117] "RemoveContainer" containerID="5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"
Sep 10 14:44:59.945083 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:44:59.945061    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961\": container with ID starting with 5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961 not found: ID does not exist" containerID="5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"
Sep 10 14:44:59.945135 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.945092    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961"} err="failed to get container status \"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961\": rpc error: code = NotFound desc = could not find container \"5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961\": container with ID starting with 5583f93d07decea2bffba51fe9b121ceb9f1726c5744543dd19f4b6fdfdfb961 not found: ID does not exist"
Sep 10 14:44:59.960061 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.960034    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:44:59.963242 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:44:59.963222    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-764f45744-tlz29"]
Sep 10 14:45:00.530930 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.530895    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" path="/var/lib/kubelet/pods/cfc8335c-3f3a-414d-a00a-7821ed40fdb8/volumes"
Sep 10 14:45:00.723522 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.723479    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764f45744-lg7vs"]
Sep 10 14:45:00.723948 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.723929    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" containerName="manager"
Sep 10 14:45:00.724059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.723950    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" containerName="manager"
Sep 10 14:45:00.724059 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.724026    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc8335c-3f3a-414d-a00a-7821ed40fdb8" containerName="manager"
Sep 10 14:45:00.728580 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.728562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.732981 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.732960    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t8r42\""
Sep 10 14:45:00.733095 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.732987    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:45:00.733095 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.733027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:45:00.733227 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.733127    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:45:00.738206 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.738168    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-lg7vs"]
Sep 10 14:45:00.804298 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.804209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w26dr\" (UniqueName: \"kubernetes.io/projected/cebb059a-ce70-4be0-859b-bd6d7a4f2734-kube-api-access-w26dr\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.804298 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.804274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-metrics-certs\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.804482 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.804342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-webhook-cert\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.905806 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.905772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-metrics-certs\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.905965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.905813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-webhook-cert\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.905965 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.905854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w26dr\" (UniqueName: \"kubernetes.io/projected/cebb059a-ce70-4be0-859b-bd6d7a4f2734-kube-api-access-w26dr\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.908429 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.908401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-metrics-certs\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.908548 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.908411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cebb059a-ce70-4be0-859b-bd6d7a4f2734-webhook-cert\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:00.915015 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:00.914993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w26dr\" (UniqueName: \"kubernetes.io/projected/cebb059a-ce70-4be0-859b-bd6d7a4f2734-kube-api-access-w26dr\") pod \"maas-controller-764f45744-lg7vs\" (UID: \"cebb059a-ce70-4be0-859b-bd6d7a4f2734\") " pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:01.040136 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:01.040101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:01.313158 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:01.313124    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f45744-lg7vs"]
Sep 10 14:45:01.316683 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:45:01.316655    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcebb059a_ce70_4be0_859b_bd6d7a4f2734.slice/crio-4a0c85a32b1bef3968c66d06f2bd0cb4874f76cdf99a1baac0e0d1db325bb9be WatchSource:0}: Error finding container 4a0c85a32b1bef3968c66d06f2bd0cb4874f76cdf99a1baac0e0d1db325bb9be: Status 404 returned error can't find the container with id 4a0c85a32b1bef3968c66d06f2bd0cb4874f76cdf99a1baac0e0d1db325bb9be
Sep 10 14:45:01.950790 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:01.950756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-lg7vs" event={"ID":"cebb059a-ce70-4be0-859b-bd6d7a4f2734","Type":"ContainerStarted","Data":"4a0c85a32b1bef3968c66d06f2bd0cb4874f76cdf99a1baac0e0d1db325bb9be"}
Sep 10 14:45:02.957197 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:02.957160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f45744-lg7vs" event={"ID":"cebb059a-ce70-4be0-859b-bd6d7a4f2734","Type":"ContainerStarted","Data":"5b94e2eaa81742453ab4deb783dbfedd5238d96f00efad6871c6b9f78ab9fb36"}
Sep 10 14:45:02.957602 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:02.957220    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:45:02.976337 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:02.976286    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-764f45744-lg7vs" podStartSLOduration=2.382627301 podStartE2EDuration="2.976271333s" totalImagesPullingTime="593.644032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:45:02.974471765 +0000 UTC m=+2014.935981280" watchObservedRunningTime="2026-09-10 14:45:02.976271333 +0000 UTC m=+2014.937780851"
Sep 10 14:45:13.964526 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:45:13.964492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764f45744-lg7vs"
Sep 10 14:46:28.608044 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:46:28.608012    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:46:28.618380 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:46:28.618355    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:48:29.648431 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.648396    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"]
Sep 10 14:48:29.652134 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.652110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.657035 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.656999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:48:29.668749 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.668695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"]
Sep 10 14:48:29.747744 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.747689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5142cb10-c071-49d6-8633-4749270b9e18-openshift-service-ca\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.747908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.747785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-metrics-certs\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.747908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.747819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlbfg\" (UniqueName: \"kubernetes.io/projected/5142cb10-c071-49d6-8633-4749270b9e18-kube-api-access-dlbfg\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.747908 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.747849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-maas-api-tls\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.848625 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.848596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5142cb10-c071-49d6-8633-4749270b9e18-openshift-service-ca\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.848794 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.848644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-metrics-certs\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.848794 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.848748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlbfg\" (UniqueName: \"kubernetes.io/projected/5142cb10-c071-49d6-8633-4749270b9e18-kube-api-access-dlbfg\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.848881 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.848795    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-maas-api-tls\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.849278 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.849256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5142cb10-c071-49d6-8633-4749270b9e18-openshift-service-ca\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.851159 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.851135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-maas-api-tls\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.851253 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.851176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5142cb10-c071-49d6-8633-4749270b9e18-metrics-certs\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.856270 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.856247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlbfg\" (UniqueName: \"kubernetes.io/projected/5142cb10-c071-49d6-8633-4749270b9e18-kube-api-access-dlbfg\") pod \"maas-api-c697fdc64-7d7d4\" (UID: \"5142cb10-c071-49d6-8633-4749270b9e18\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:29.962694 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:29.962623    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:30.238580 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:30.238509    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"]
Sep 10 14:48:30.241957 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:48:30.241931    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5142cb10_c071_49d6_8633_4749270b9e18.slice/crio-53ea72e0f4021f9371cf80836c6e3d8b65f82b408b9daa4db994c0ca0f5da842 WatchSource:0}: Error finding container 53ea72e0f4021f9371cf80836c6e3d8b65f82b408b9daa4db994c0ca0f5da842: Status 404 returned error can't find the container with id 53ea72e0f4021f9371cf80836c6e3d8b65f82b408b9daa4db994c0ca0f5da842
Sep 10 14:48:30.917039 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:30.916959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4" event={"ID":"5142cb10-c071-49d6-8633-4749270b9e18","Type":"ContainerStarted","Data":"9d158d90fabf8dcc3a4ed476e489a9bcd3b612a320dbbde510161e890d7e2e80"}
Sep 10 14:48:30.917039 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:30.917012    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4" event={"ID":"5142cb10-c071-49d6-8633-4749270b9e18","Type":"ContainerStarted","Data":"53ea72e0f4021f9371cf80836c6e3d8b65f82b408b9daa4db994c0ca0f5da842"}
Sep 10 14:48:30.917426 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:30.917049    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:30.934893 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:30.934837    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4" podStartSLOduration=1.538462785 podStartE2EDuration="1.934822297s" totalImagesPullingTime="396.359512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:30.933910809 +0000 UTC m=+2222.895420327" watchObservedRunningTime="2026-09-10 14:48:30.934822297 +0000 UTC m=+2222.896331815"
Sep 10 14:48:36.927838 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:36.927809    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c697fdc64-7d7d4"
Sep 10 14:48:36.985940 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:36.985901    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:48:36.986230 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:36.986202    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" podUID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" containerName="maas-api" containerID="cri-o://0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a" gracePeriod=30
Sep 10 14:48:38.227259 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.227238    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:48:38.324645 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.324572    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") pod \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") "
Sep 10 14:48:38.324645 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.324605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs\") pod \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") "
Sep 10 14:48:38.324645 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.324632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca\") pod \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") "
Sep 10 14:48:38.324943 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.324669    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9xqs\" (UniqueName: \"kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs\") pod \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\" (UID: \"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8\") "
Sep 10 14:48:38.325026 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.324996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" (UID: "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:48:38.326794 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.326771    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" (UID: "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:38.326900 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.326790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" (UID: "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:38.326900 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.326859    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs" (OuterVolumeSpecName: "kube-api-access-b9xqs") pod "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" (UID: "3b5d1d7a-1e3e-4324-a6f0-57747afcefa8"). InnerVolumeSpecName "kube-api-access-b9xqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:38.425603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.425576    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-openshift-service-ca\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:48:38.425603 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.425601    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9xqs\" (UniqueName: \"kubernetes.io/projected/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-kube-api-access-b9xqs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:48:38.425813 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.425610    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-maas-api-tls\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:48:38.425813 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.425619    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8-metrics-certs\") on node \"ip-10-0-131-209.ec2.internal\" DevicePath \"\""
Sep 10 14:48:38.954616 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.954583    2581 generic.go:356] "Generic (PLEG): container finished" podID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" containerID="0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a" exitCode=0
Sep 10 14:48:38.954821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.954624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"
Sep 10 14:48:38.954821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.954628    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" event={"ID":"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8","Type":"ContainerDied","Data":"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"}
Sep 10 14:48:38.954821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.954760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv" event={"ID":"3b5d1d7a-1e3e-4324-a6f0-57747afcefa8","Type":"ContainerDied","Data":"ab8d60b322fbbc472be15c6503d138db5acaa271119ab7ce0803cb36311aa1b9"}
Sep 10 14:48:38.954821 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.954792    2581 scope.go:117] "RemoveContainer" containerID="0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"
Sep 10 14:48:38.963601 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.963573    2581 scope.go:117] "RemoveContainer" containerID="0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"
Sep 10 14:48:38.963884 ip-10-0-131-209 kubenswrapper[2581]: E0910 14:48:38.963865    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a\": container with ID starting with 0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a not found: ID does not exist" containerID="0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"
Sep 10 14:48:38.963939 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.963893    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a"} err="failed to get container status \"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a\": rpc error: code = NotFound desc = could not find container \"0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a\": container with ID starting with 0c25a0d8a4ee7757f06d108c4a652dde6635e48063df64c71571fa20be28f16a not found: ID does not exist"
Sep 10 14:48:38.970428 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.970397    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:48:38.977134 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:38.977115    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-8nzwv"]
Sep 10 14:48:40.531771 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:40.531736    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" path="/var/lib/kubelet/pods/3b5d1d7a-1e3e-4324-a6f0-57747afcefa8/volumes"
Sep 10 14:48:59.392152 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:48:59.392069    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685577bfc4-mwhhg_804fa8e8-d78f-49ce-b312-4aefbafb8169/authorino/0.log"
Sep 10 14:49:04.750054 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:04.750019    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764f45744-lg7vs_cebb059a-ce70-4be0-859b-bd6d7a4f2734/manager/0.log"
Sep 10 14:49:05.108995 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:05.108964    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-72wk2_70c591ef-cbaf-4bdf-8b23-3e92d8d51876/manager/0.log"
Sep 10 14:49:05.115175 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:05.115152    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-72wk2_70c591ef-cbaf-4bdf-8b23-3e92d8d51876/copy-manifests/0.log"
Sep 10 14:49:05.771222 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:05.771188    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c697fdc64-7d7d4_5142cb10-c071-49d6-8633-4749270b9e18/maas-api/0.log"
Sep 10 14:49:05.880584 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:05.880558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817510-dfnfs_7c621520-44ce-476a-bba5-9282a5b0f486/cleanup/0.log"
Sep 10 14:49:06.102099 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:06.102011    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bf7sq_4f8c8d38-afbe-478c-84a0-432a4cb1647f/postgres/0.log"
Sep 10 14:49:06.838962 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:06.838933    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685577bfc4-mwhhg_804fa8e8-d78f-49ce-b312-4aefbafb8169/authorino/0.log"
Sep 10 14:49:08.034658 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:08.034627    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-rctfp_57186202-2dcf-40b5-bc92-a2d99fcb8771/kube-auth-proxy/0.log"
Sep 10 14:49:08.388312 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:08.388231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spm9g_b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8/payload-pre-processing/0.log"
Sep 10 14:49:09.052778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:09.052747    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5mc7r_ceb37ea6-9ab3-4a5f-8599-0b5e827fb5f9/main/0.log"
Sep 10 14:49:09.155755 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:09.155727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-4bksv_cb00c14b-cc97-475b-afc9-399fbed512c2/main/0.log"
Sep 10 14:49:09.469376 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:09.469296    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-p6stq_ba573bee-2de2-4e82-ad2f-60697ee7a8cb/main/0.log"
Sep 10 14:49:18.383015 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:18.382980    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8rqck_a4090977-ff09-4a16-bed8-f4d8ba6a4efa/global-pull-secret-syncer/0.log"
Sep 10 14:49:18.533567 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:18.533533    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vp8wd_af45afff-ba25-4cfb-82dd-caabe513b30a/konnectivity-agent/0.log"
Sep 10 14:49:18.578971 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:18.578936    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-209.ec2.internal_be1a10a3d481e4ebb21a9b9822808141/haproxy/0.log"
Sep 10 14:49:22.662866 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:22.662830    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685577bfc4-mwhhg_804fa8e8-d78f-49ce-b312-4aefbafb8169/authorino/0.log"
Sep 10 14:49:24.710456 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.710422    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-7xq78_0f425bdc-f513-4b90-8fb3-3ebb7595f6a5/cluster-monitoring-operator/0.log"
Sep 10 14:49:24.740256 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.740214    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-zb668_8057d5a6-5443-4139-962d-e10a82f3b8f2/kube-state-metrics/0.log"
Sep 10 14:49:24.761318 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.761284    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-zb668_8057d5a6-5443-4139-962d-e10a82f3b8f2/kube-rbac-proxy-main/0.log"
Sep 10 14:49:24.783599 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.783523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-zb668_8057d5a6-5443-4139-962d-e10a82f3b8f2/kube-rbac-proxy-self/0.log"
Sep 10 14:49:24.850610 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.850583    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-8mcmh_d183bbac-4cdd-4e24-b3fd-bee5b2b15f47/monitoring-plugin/0.log"
Sep 10 14:49:24.895574 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.895538    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sv55_3d5c392b-926a-481f-b7ae-f069a4d6b908/node-exporter/0.log"
Sep 10 14:49:24.916598 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.916570    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sv55_3d5c392b-926a-481f-b7ae-f069a4d6b908/kube-rbac-proxy/0.log"
Sep 10 14:49:24.939488 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:24.939460    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sv55_3d5c392b-926a-481f-b7ae-f069a4d6b908/init-textfile/0.log"
Sep 10 14:49:25.118520 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.118494    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-26lv9_2886062c-df86-4d79-a826-2d4172bc0c45/kube-rbac-proxy-main/0.log"
Sep 10 14:49:25.141154 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.141127    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-26lv9_2886062c-df86-4d79-a826-2d4172bc0c45/kube-rbac-proxy-self/0.log"
Sep 10 14:49:25.160467 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.160441    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-26lv9_2886062c-df86-4d79-a826-2d4172bc0c45/openshift-state-metrics/0.log"
Sep 10 14:49:25.197892 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.197858    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/prometheus/0.log"
Sep 10 14:49:25.217094 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.217071    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/config-reloader/0.log"
Sep 10 14:49:25.238251 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.238224    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/thanos-sidecar/0.log"
Sep 10 14:49:25.259397 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.259365    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/kube-rbac-proxy-web/0.log"
Sep 10 14:49:25.281331 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.281310    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/kube-rbac-proxy/0.log"
Sep 10 14:49:25.306623 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.306543    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/kube-rbac-proxy-thanos/0.log"
Sep 10 14:49:25.330367 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.330346    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6dde411f-5379-41ab-a182-8502fdde692f/init-config-reloader/0.log"
Sep 10 14:49:25.416138 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:25.416111    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-gbxjz_e17d8a37-068e-4ee1-ad53-05b4fe22f1cd/prometheus-operator-admission-webhook/0.log"
Sep 10 14:49:27.372313 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:27.372272    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-z4gmp_52422580-bfd8-4fec-b599-ab0a12145f18/networking-console-plugin/0.log"
Sep 10 14:49:28.626433 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:28.626403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-2dlw6_ed9bbea1-85c9-4de0-b61b-50813c8d03e6/download-server/0.log"
Sep 10 14:49:29.599802 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.599765    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"]
Sep 10 14:49:29.600153 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.600140    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" containerName="maas-api"
Sep 10 14:49:29.600153 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.600153    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" containerName="maas-api"
Sep 10 14:49:29.600293 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.600269    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5d1d7a-1e3e-4324-a6f0-57747afcefa8" containerName="maas-api"
Sep 10 14:49:29.604948 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.604931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.607409 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.607383    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cvj6w\"/\"default-dockercfg-k2q7n\""
Sep 10 14:49:29.608575 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.608555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cvj6w\"/\"openshift-service-ca.crt\""
Sep 10 14:49:29.608661 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.608565    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cvj6w\"/\"kube-root-ca.crt\""
Sep 10 14:49:29.613008 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.612988    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"]
Sep 10 14:49:29.660505 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.660463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-proc\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.660994 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.660512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xtbd\" (UniqueName: \"kubernetes.io/projected/4306f5ce-2d61-4f58-af20-567fafb4b840-kube-api-access-5xtbd\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.660994 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.660544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-lib-modules\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.660994 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.660646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-sys\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.660994 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.660683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-podres\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.761624 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-sys\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.761624 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-podres\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-proc\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-sys\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xtbd\" (UniqueName: \"kubernetes.io/projected/4306f5ce-2d61-4f58-af20-567fafb4b840-kube-api-access-5xtbd\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-lib-modules\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-proc\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-podres\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.762013 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.761876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4306f5ce-2d61-4f58-af20-567fafb4b840-lib-modules\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.770275 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.770246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xtbd\" (UniqueName: \"kubernetes.io/projected/4306f5ce-2d61-4f58-af20-567fafb4b840-kube-api-access-5xtbd\") pod \"perf-node-gather-daemonset-fqw74\" (UID: \"4306f5ce-2d61-4f58-af20-567fafb4b840\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:29.916180 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:29.916086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:30.185884 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:30.185860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"]
Sep 10 14:49:30.188204 ip-10-0-131-209 kubenswrapper[2581]: W0910 14:49:30.188172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4306f5ce_2d61_4f58_af20_567fafb4b840.slice/crio-b3ca233b9b3dc57a5b0efce3e3499b0633837d7d267a43ab46947644b43639c3 WatchSource:0}: Error finding container b3ca233b9b3dc57a5b0efce3e3499b0633837d7d267a43ab46947644b43639c3: Status 404 returned error can't find the container with id b3ca233b9b3dc57a5b0efce3e3499b0633837d7d267a43ab46947644b43639c3
Sep 10 14:49:30.194740 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:30.194719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74" event={"ID":"4306f5ce-2d61-4f58-af20-567fafb4b840","Type":"ContainerStarted","Data":"b3ca233b9b3dc57a5b0efce3e3499b0633837d7d267a43ab46947644b43639c3"}
Sep 10 14:49:30.555401 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:30.555371    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hd5g_68e8319c-15ef-4012-b590-a364beff74e7/dns/0.log"
Sep 10 14:49:30.579610 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:30.579580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hd5g_68e8319c-15ef-4012-b590-a364beff74e7/kube-rbac-proxy/0.log"
Sep 10 14:49:30.712549 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:30.712520    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96r9b_2a767a03-df9e-4854-b713-d4d427d181da/dns-node-resolver/0.log"
Sep 10 14:49:31.200578 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:31.200541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74" event={"ID":"4306f5ce-2d61-4f58-af20-567fafb4b840","Type":"ContainerStarted","Data":"34044b19f2bb067bcf39bfa365daa7920b13afa61dce26daaba014bc0a587812"}
Sep 10 14:49:31.200789 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:31.200639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:31.217581 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:31.217530    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74" podStartSLOduration=2.217514947 podStartE2EDuration="2.217514947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:31.216238484 +0000 UTC m=+2283.177748002" watchObservedRunningTime="2026-09-10 14:49:31.217514947 +0000 UTC m=+2283.179024464"
Sep 10 14:49:31.381444 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:31.381409    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmknf_1e6dbcd2-d378-4917-a184-0dd655be1320/node-ca/0.log"
Sep 10 14:49:32.581026 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:32.580998    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-rctfp_57186202-2dcf-40b5-bc92-a2d99fcb8771/kube-auth-proxy/0.log"
Sep 10 14:49:32.689499 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:32.689465    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spm9g_b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8/payload-pre-processing/0.log"
Sep 10 14:49:33.641160 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:33.641125    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx4k4_1c34ec40-6ca5-4fe5-8a19-ed1fdc1770f4/serve-healthcheck-canary/0.log"
Sep 10 14:49:34.135951 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:34.135917    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-22b5c_1c283e68-f138-4300-8e04-bb7fb2727d30/insights-operator/0.log"
Sep 10 14:49:34.136353 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:34.136328    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-22b5c_1c283e68-f138-4300-8e04-bb7fb2727d30/insights-operator/1.log"
Sep 10 14:49:34.296276 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:34.296246    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xzzw5_cd51d5fd-2863-44ce-82fc-f1da8a53857b/kube-rbac-proxy/0.log"
Sep 10 14:49:34.315723 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:34.315679    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xzzw5_cd51d5fd-2863-44ce-82fc-f1da8a53857b/exporter/0.log"
Sep 10 14:49:34.339733 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:34.339683    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xzzw5_cd51d5fd-2863-44ce-82fc-f1da8a53857b/extractor/0.log"
Sep 10 14:49:36.780498 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:36.780458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764f45744-lg7vs_cebb059a-ce70-4be0-859b-bd6d7a4f2734/manager/0.log"
Sep 10 14:49:37.002573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:37.002539    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-72wk2_70c591ef-cbaf-4bdf-8b23-3e92d8d51876/manager/0.log"
Sep 10 14:49:37.019540 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:37.019519    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-72wk2_70c591ef-cbaf-4bdf-8b23-3e92d8d51876/copy-manifests/0.log"
Sep 10 14:49:37.215492 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:37.215413    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-fqw74"
Sep 10 14:49:38.674818 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:38.674790    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b64d8b6d6-vl696_d62a7918-bfdb-4bfc-bb18-0b66a31891c8/manager/0.log"
Sep 10 14:49:39.009551 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:39.009519    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-rctfp_57186202-2dcf-40b5-bc92-a2d99fcb8771/kube-auth-proxy/0.log"
Sep 10 14:49:39.102204 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:39.102167    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spm9g_b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8/payload-pre-processing/0.log"
Sep 10 14:49:40.968996 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:40.968913    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-rctfp_57186202-2dcf-40b5-bc92-a2d99fcb8771/kube-auth-proxy/0.log"
Sep 10 14:49:41.059545 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:41.059512    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spm9g_b6f37565-3bd6-4ac5-bb56-f5d3a8f45ec8/payload-pre-processing/0.log"
Sep 10 14:49:45.287796 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.287758    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7fdm7_64bf7fd1-d479-4272-a4e9-d9e5a950d89e/kube-multus/0.log"
Sep 10 14:49:45.639887 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.639810    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/kube-multus-additional-cni-plugins/0.log"
Sep 10 14:49:45.660613 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.660582    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/egress-router-binary-copy/0.log"
Sep 10 14:49:45.682778 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.682753    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/cni-plugins/0.log"
Sep 10 14:49:45.704030 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.704009    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/bond-cni-plugin/0.log"
Sep 10 14:49:45.726195 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.726169    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/routeoverride-cni/0.log"
Sep 10 14:49:45.750573 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.750544    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/whereabouts-cni-bincopy/0.log"
Sep 10 14:49:45.776465 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.776439    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkbrc_858c0a41-f465-4cac-b5e5-8273e9bac7b3/whereabouts-cni/0.log"
Sep 10 14:49:45.893247 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.893154    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6ghsl_b9c4139c-2dde-4cd3-ba84-c9bee977f4ea/network-metrics-daemon/0.log"
Sep 10 14:49:45.911914 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:45.911889    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6ghsl_b9c4139c-2dde-4cd3-ba84-c9bee977f4ea/kube-rbac-proxy/0.log"
Sep 10 14:49:47.109539 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.109497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-controller/0.log"
Sep 10 14:49:47.125497 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.125472    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/1.log"
Sep 10 14:49:47.125663 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.125477    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovn-acl-logging/0.log"
Sep 10 14:49:47.150095 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.150074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/kube-rbac-proxy-node/0.log"
Sep 10 14:49:47.173288 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.173229    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 14:49:47.190481 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.190457    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/northd/0.log"
Sep 10 14:49:47.210859 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.210838    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/nbdb/0.log"
Sep 10 14:49:47.232032 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.231996    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/sbdb/0.log"
Sep 10 14:49:47.319715 ip-10-0-131-209 kubenswrapper[2581]: I0910 14:49:47.319678    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jnxx6_a2f2bcda-a926-40e6-a7c2-73ef65ed0602/ovnkube-controller/0.log"